{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kannada MNIST Digits Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is my second iteration of the Kannada MNIST dataset I found on Kaggle. I decided to start from scratch because I wanted to try using a CNN and implementing data augmentation. \n",
    "\n",
    "My previous attempt resulted in accuracy scores in the 90%s on the validation set; however, I received accuracy scores in the 60%s on the holdout set. This made me think that I either didn't have enough data to train on, or the data in the holdout set wasn't an accuracte representation of the training data. My previous notebook compared neural network of just Dense layers (2x50, 2x100, 3x50, and 3x100).\n",
    "\n",
    "Since this is a dataset on digits, I think I will be able to improve accuracy scores by using a CNN and augmenting the data through rotations, shearing, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing and Reshaping Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "# Import Packages\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from keras.utils import to_categorical, np_utils\n",
    "from sklearn.utils import shuffle\n",
    "from keras import regularizers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>pixel0</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel774</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   label  pixel0  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  \\\n",
       "0      0       0       0       0       0       0       0       0       0   \n",
       "1      1       0       0       0       0       0       0       0       0   \n",
       "2      2       0       0       0       0       0       0       0       0   \n",
       "3      3       0       0       0       0       0       0       0       0   \n",
       "4      4       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel8  ...  pixel774  pixel775  pixel776  pixel777  pixel778  pixel779  \\\n",
       "0       0  ...         0         0         0         0         0         0   \n",
       "1       0  ...         0         0         0         0         0         0   \n",
       "2       0  ...         0         0         0         0         0         0   \n",
       "3       0  ...         0         0         0         0         0         0   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "\n",
       "   pixel780  pixel781  pixel782  pixel783  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2         0         0         0         0  \n",
       "3         0         0         0         0  \n",
       "4         0         0         0         0  \n",
       "\n",
       "[5 rows x 785 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import Data\n",
    "df = pd.read_csv('train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Convert features and target into arrays\n",
    "y = df['label'].values\n",
    "X = df.loc[:,df.columns != 'label'].values\n",
    "X = np.true_divide(X, 255)\n",
    "\n",
    "# Make n_cols variable to use in model parameters\n",
    "n_cols = X.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split data into train and test\n",
    "from sklearn.model_selection import train_test_split\n",
    "Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size = 0.3, random_state = 56)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMyElEQVR4nO3db4hd9Z3H8c9nbfokzYNETSbasNYScFelqQZZsCxKacn6JKlQbR4sWf9N1UQT8IGiSIRSDNJ2WcQUpipN1qyhEMWh1KYSwuoKBkfJ6qRpqyuzzZ8h8c+D2kdR57sP5swyxrnnjvecc891vu8XDPfe873nni+HfPI75557788RIQAL39+03QCA/iDsQBKEHUiCsANJEHYgiS/1c2O2eesfaFhEeK7llUZ22+ts/9H227bvq/JaAJrlXq+z2z5H0p8kfUfScUmvStoYEb8vWYeRHWhYEyP7VZLejoh3IuKMpL2S1ld4PQANqhL2CyUdm/X4eLHsU2wP2x6zPVZhWwAqqvIG3VyHCp85TI+IEUkjEofxQJuqjOzHJa2a9firkk5WawdAU6qE/VVJq21/zfaXJf1A0mg9bQGoW8+H8RHxse0tkvZLOkfSkxFxpLbOANSq50tvPW2Mc3agcY18qAbAFwdhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSfQ8ZTMWhvPPP7+0fuRI+Szc3dYvs3PnztL65s2be35tfFalsNuekPShpE8kfRwRa+toCkD96hjZr42I92p4HQAN4pwdSKJq2EPS72y/Znt4rifYHrY9Znus4rYAVFD1MP7qiDhpe7mkF2z/ISJenP2EiBiRNCJJtqPi9gD0qNLIHhEni9vTkp6VdFUdTQGoX89ht73Y9pKZ+5K+K2m8rsYA1KvKYfwKSc/annmd/4iI39bSFVK4/vrrS+tTU1Ol9bvuuqvOdha8nsMeEe9I+kaNvQBoEJfegCQIO5AEYQeSIOxAEoQdSIKvuC5wl1xySWl9+/btpfUlS5bU2c6nDA0NldYvu+yyxradESM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiThiP79eAy/VNOMK664omPt9ttvL133tttuq7Ttp556qrR+7NixjrWyviVp9erVpfXHH3+8tP7www+X1heqiPBcyxnZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJvs/+BbBmzZrSetm19A0bNpSuu2/fvp56mrFjx47SetmUz+vWrStd99Zbby2tX3nllaV1fBojO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwffZvwAee+yx0vqNN97YsTY6Olq67s0339xTT3U499xzS+vdfrN+YmKixm4Wjp6/z277SdunbY/PWrbM9gu23ypul9bZLID6zecw/peSzv6o032SDkTEakkHiscABljXsEfEi5I+OGvxekm7ivu7JJV/JhNA63r9bPyKiJiUpIiYtL280xNtD0sa7nE7AGrS+BdhImJE0ojEG3RAm3q99HbK9kpJKm5P19cSgCb0GvZRSZuK+5skPVdPOwCa0vUw3vbTkq6RdJ7t45K2S9oh6Ve2b5H0Z0nfb7LJhW7RokWl9ampqdL6nj17Ota2bt3aU0/9cMMNN1SqX3vttXW2s+B1DXtEbOxQ+nbNvQBoEB+XBZIg7EAShB1IgrADSRB2IAl+SnoA7N69u7T+yiuvlNa7TZsMSIzsQBqEHUiCsANJEHYgCcIOJEHYgSQIO5AE19kHwNDQUGn9zJkzpfX333+/znawQDGyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASXGfvg0cffbS0/vzzz5fWX3rppTrbQVKM7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBNfZ+2DLli2l9W5TD4+Pj9fZDpLqOrLbftL2advjs5Y9ZPuE7cPF33XNtgmgqvkcxv9S0ro5lv9rRKwp/n5Tb1sA6tY17BHxoqQP+tALgAZVeYNui+03isP8pZ2eZHvY9pjtsQrbAlBRr2H/uaSvS1ojaVLSTzs9MSJGImJtRKztcVsAatBT2CPiVER8EhFTkn4h6ap62wJQt57CbnvlrIffk8S1IWDAdb3ObvtpSddIOs/2cUnbJV1je42kkDQh6YcN9gigBl3DHhEb51j8RAO9AGgQH5cFkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAmmbO6Dyy+/vLT+4IMPltaHhoZK63v37v3cPfXLAw880LG2bNmy0nVvuummuttJjZEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgOnsfjI+XT1+/fPny0vrSpUvrbKevLrjggp7XnZiYqK8RdB/Zba+yfdD2UdtHbG8tli+z/YLtt4rbL+6/SCCB+RzGfyzpnoj4O0n/IGmz7b+XdJ+kAxGxWtKB4jGAAdU17BExGRGvF/c/lHRU0oWS1kvaVTxtl6QNTTUJoLrPdc5u+yJJ35R0SNKKiJiUpv9DsD3niaftYUnD1doEUNW8w277K5L2SdoWEX+xPa/1ImJE0kjxGtFLkwCqm9elN9uLNB30PRHxTLH4lO2VRX2lpNPNtAigDl1Hdk8P4U9IOhoRP5tVGpW0SdKO4va5RjqE1q9fX1o/ceJEx9ro6GjpuosXLy6tb9u2rdL6p06d6lg7dOhQ6bqo13wO46+W9M+S3rR9uFh2v6ZD/ivbt0j6s6TvN9MigDp0DXtE/JekTifo3663HQBN4eOyQBKEHUiCsANJEHYgCcIOJMFXXAfA/v37S+ubNm0qrd99990da91+rnnJkiWl9Xvvvbe0/vLLL5fWH3nkkY61gwcPlq6LejGyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASjujfj8fwSzW9ueeee0rrd955Z8faxRdfXLruRx99VFo/fPhwaX14uPwXx7qtj/pFxJzfUmVkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkuM6+ANxxxx0dazt37ixd99133y2tX3rppZXWR/9xnR1IjrADSRB2IAnCDiRB2IEkCDuQBGEHkuh6nd32Kkm7JQ1JmpI0EhH/ZvshSbdJmrnQen9E/KbLa3GdHWhYp+vs8wn7SkkrI+J120skvSZpg6QbJP01In4y3yYIO9C8TmGfz/zsk5Imi/sf2j4q6cJ62wPQtM91zm77IknflHSoWLTF9hu2n7S9tMM6w7bHbI9V6hRAJfP+bLztr0j6T0k/johnbK+Q9J6kkPQjTR/q39zlNTiMBxrW8zm7JNleJOnXkvZHxM/mqF8k6dcRcVmX1yHsQMN6/iKMbUt6QtLR2UEv3rib8T1J41WbBNCc+bwb/y1JL0l6U9OX3iTpfkkbJa3R9GH8hKQfFm/mlb0WIzvQsEqH8XUh7EDz+D47kBxhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgia4/OFmz9yT976zH5xXLBtGg9jaofUn01qs6e/vbToW+fp/9Mxu3xyJibWsNlBjU3ga1L4neetWv3jiMB5Ig7EASbYd9pOXtlxnU3ga1L4neetWX3lo9ZwfQP22P7AD6hLADSbQSdtvrbP/R9tu272ujh05sT9h+0/bhtuenK+bQO217fNayZbZfsP1WcTvnHHst9faQ7RPFvjts+7qWeltl+6Dto7aP2N5aLG9135X01Zf91vdzdtvnSPqTpO9IOi7pVUkbI+L3fW2kA9sTktZGROsfwLD9j5L+Kmn3zNRath+R9EFE7Cj+o1waEfcOSG8P6XNO491Qb52mGf8Xtbjv6pz+vBdtjOxXSXo7It6JiDOS9kpa30IfAy8iXpT0wVmL10vaVdzfpel/LH3XobeBEBGTEfF6cf9DSTPTjLe670r66os2wn6hpGOzHh/XYM33HpJ+Z/s128NtNzOHFTPTbBW3y1vu52xdp/Hup7OmGR+YfdfL9OdVtRH2uaamGaTrf1dHxBWS/knS5uJwFfPzc0lf1/QcgJOSftpmM8U04/skbYuIv7TZy2xz9NWX/dZG2I9LWjXr8VclnWyhjzlFxMni9rSkZzV92jFITs3MoFvcnm65n/8XEaci4pOImJL0C7W474ppxvdJ2hMRzxSLW993c/XVr/3WRthflbTa9tdsf1nSDySNttDHZ9heXLxxItuLJX1XgzcV9aikTcX9TZKea7GXTxmUabw7TTOulvdd69OfR0Tf/yRdp+l35P9H0gNt9NChr4sl/Xfxd6Tt3iQ9renDuo80fUR0i6RzJR2Q9FZxu2yAevt3TU/t/Yamg7Wypd6+pelTwzckHS7+rmt735X01Zf9xsdlgST4BB2QBGEHkiDsQBKEHUiCsANJEHYgCcIOJPF/0Fzl+qw4hrAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAALzUlEQVR4nO3dX4xU9RnG8eepulyoSQED3SIWarxDiw3hwppG0kgoN2iMjVzRtOl6URub9EKkF5I0/klTNb0ywUjExmpMwMqFKRJjxAtjWAjlj0ShslVkwxZp0/UGqr692EOzws7Mcs6ZObP7fj/JZGbOmTnnnTP77Pmdf/NzRAjA7PeNpgsA0BuEHUiCsANJEHYgCcIOJHFlL2dmm13/QJdFhKcaXmnNbnuN7Q9sH7e9scq0AHSXyx5nt32FpA8l3SnppKS9ktZHxPtt3sOaHeiybqzZV0o6HhEfRcR5SS9LWldhegC6qErYF0n6ZNLzk8Wwr7E9ZHvY9nCFeQGoqMoOuqmaCpc00yNii6QtEs14oElV1uwnJS2e9Px6SaeqlQOgW6qEfa+km2wvtT0g6T5JO+spC0DdSjfjI+IL2w9I2iXpCklbI+JIbZUBqFXpQ2+lZsY2O9B1XTmpBsDMQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IonT/7JJke0TSuKQvJX0RESvqKApA/SqFvbAqIs7UMB0AXUQzHkiiathD0hu299kemuoFtodsD9serjgvABU4Isq/2f52RJyyvUDSbkm/iog9bV5ffmYApiUiPNXwSmv2iDhV3I9JelXSyirTA9A9pcNu+2rb1154LGm1pMN1FQagXlX2xi+U9KrtC9P5c0T8tZaq0DcGBgbajj937lzpac+ZM6ft+PPnz5eeNi5VOuwR8ZGk79VYC4Au4tAbkARhB5Ig7EAShB1IgrADSdRxIQxmsPnz57cdf+ZM+2ucPvvss9Lz7nTYrjisi5qwZgeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJCr9Us1lz4xfquk7Vb//KsfCT5w40Xb8qlWr2o4fGRkpPe/ZrCu/VANg5iDsQBKEHUiCsANJEHYgCcIOJEHYgSS4nn2WW7ZsWaX3Hzp0qKZKLrV06dK24zudA8D17peHNTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMH17LNcp+/33XffbTv+tttuq7Ocy8Jx9nJKX89ue6vtMduHJw2bZ3u37WPF/dw6iwVQv+k045+XtOaiYRslvRkRN0l6s3gOoI91DHtE7JF09qLB6yRtKx5vk3RXzXUBqFnZc+MXRsSoJEXEqO0FrV5oe0jSUMn5AKhJ1y+EiYgtkrZI7KADmlT20Ntp24OSVNyP1VcSgG4oG/adkjYUjzdIeq2ecgB0S8fj7LZfknSHpOsknZb0iKS/SHpF0g2SPpZ0b0RcvBNvqmnRjO+C1atXtxy3a9eutu/t52PVHGcvp9Vx9o7b7BGxvsWoH1WqCEBPcboskARhB5Ig7EAShB1IgrADSfBT0rNAu8Nr27dv72ElvXXPPfe0HT+bP3sZrNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAl+SnoWaPcdzuTLQKv+bc7kz15F6Z+SBjA7EHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAS/G78DPDwww+3Hf/444/3qBLMZB3X7La32h6zfXjSsM22P7V9oLit7W6ZAKqaTjP+eUlrphj+dEQsL26v11sWgLp1DHtE7JF0tge1AOiiKjvoHrB9sGjmz231IttDtodtD1eYF4CKyob9GUk3SlouaVTSk61eGBFbImJFRKwoOS8ANSgV9og4HRFfRsRXkp6VtLLesgDUrVTYbQ9Oenq3pMOtXgugP3Q8zm77JUl3SLrO9klJj0i6w/ZySSFpRNL9Xawxvccee6zt+Ky/j75p06amS5hROoY9ItZPMfi5LtQCoIs4XRZIgrADSRB2IAnCDiRB2IEk6LJ5Buj0Hc3WQ29ZP3dVdNkMJEfYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgiY5ht73Y9lu2j9o+YvvBYvg827ttHyvu53a/XABldewRxvagpMGI2G/7Wkn7JN0l6aeSzkbEE7Y3SpobEQ91mBY9wpSQtWeUrJ+7qtI9wkTEaETsLx6PSzoqaZGkdZK2FS/bpol/AAD61JWX82LbSyTdKuk9SQsjYlSa+Idge0GL9wxJGqpWJoCqpt2xo+1rJL0t6dGI2GH73xHxzUnj/xURbbfbacaXk7U5m/VzV1WpY0fbV0naLunFiNhRDD5dbM9f2K4fq6NQAN0xnb3xlvScpKMR8dSkUTslbSgeb5D0Wv3lAajLdPbG3y7pHUmHJH1VDN6kie32VyTdIOljSfdGxNkO06IZX0LW5mzWz11Vq2b8tLfZ60DYy8n6R5/1c1dVaZsdwMxH2IEkCDuQBGEHkiDsQBKXdbos+tPBgwdbjrvlllt6WEm9br755qZLmFVYswNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAElz1NgMsWbKk7fgTJ060HMeVYflw1RuQHGEHkiDsQBKEHUiCsANJEHYgCcIOJMFx9lmg3XfIcfZ8OM4OJEfYgSQIO5AEYQeSIOxAEoQdSIKwA0l0/N1424slvSDpW5rosnlLRPzR9mZJv5D0z+KlmyLi9W4VitbmzJnTcty5c+dKvxezy3Q6ifhC0m8iYr/tayXts727GPd0RPyhe+UBqEvHsEfEqKTR4vG47aOSFnW7MAD1uqxtdttLJN0q6b1i0AO2D9reantui/cM2R62PVypUgCVTPvceNvXSHpb0qMRscP2QklnJIWk30kajIifdZgG58Z3wcDAQMtx4+Pjbd/LNvvsU+nceNtXSdou6cWI2FFM8HREfBkRX0l6VtLKuooFUL+OYffEZVPPSToaEU9NGj446WV3Szpcf3kA6tKxGW/7dknvSDqkiUNvkrRJ0npJyzXRjB+RdH+xM6/dtGjGA13WqhnP9ezALMP17EByhB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSSm8+uydToj6R+Tnl9XDOtH/Vpbv9YlUVtZddb2nVYjeno9+yUzt4cjYkVjBbTRr7X1a10StZXVq9poxgNJEHYgiabDvqXh+bfTr7X1a10StZXVk9oa3WYH0DtNr9kB9AhhB5JoJOy219j+wPZx2xubqKEV2yO2D9k+0HT/dEUfemO2D08aNs/2btvHivsp+9hrqLbNtj8tlt0B22sbqm2x7bdsH7V9xPaDxfBGl12bunqy3Hq+zW77CkkfSrpT0klJeyWtj4j3e1pIC7ZHJK2IiMZPwLD9Q0mfS3ohIpYVw34v6WxEPFH8o5wbEQ/1SW2bJX3edDfeRW9Fg5O7GZd0l6SfqsFl16aun6gHy62JNftKSccj4qOIOC/pZUnrGqij70XEHklnLxq8TtK24vE2Tfyx9FyL2vpCRIxGxP7i8bikC92MN7rs2tTVE02EfZGkTyY9P6n+6u89JL1he5/toaaLmcLCC91sFfcLGq7nYh278e6li7oZ75tlV6b786qaCPtUXdP00/G/H0TE9yX9WNIvi+YqpucZSTdqog/AUUlPNllM0c34dkm/joj/NFnLZFPU1ZPl1kTYT0paPOn59ZJONVDHlCLiVHE/JulV9V9X1Kcv9KBb3I81XM//9VM33lN1M64+WHZNdn/eRNj3SrrJ9lLbA5Luk7SzgTouYfvqYseJbF8tabX6ryvqnZI2FI83SHqtwVq+pl+68W7VzbgaXnaNd38eET2/SVqriT3yf5f02yZqaFHXdyX9rbgdabo2SS9poln3X020iH4uab6kNyUdK+7n9VFtf9JE194HNRGswYZqu10Tm4YHJR0obmubXnZt6urJcuN0WSAJzqADkiDsQBKEHUiCsANJEHYgCcIOJEHYgST+ByFr+t8Dl0ntAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMrElEQVR4nO3dYYwc9XnH8d+PawIS8Qu7lq0ToSGNwKKqVCjGQiKqXKJEFBlMXqSKJSpXjXR5AVIi+qIolWUQFEVVk75CkS4CxanCRZHAxXeKk1hWgJQXgTOiYOLGuMhNHJ/OsvwijniRcn764sbVYe/MHDszO+t7vh/ptLvz7Mw8rPj5P7uzO39HhACsfVf13QCA0SDsQBKEHUiCsANJEHYgiT8Y5c5s89E/0LGI8KDljUZ223fb/qXtE7YfabItAN3ysOfZbU9IOi7ps5JOSXpN0q6I+EXFOozsQMe6GNm3SToREe9GxO8lfV/SzgbbA9ChJmG/TtKvVzw+VSz7ANtTtudtzzfYF4CGmnxAN+hQ4bLD9IiYljQtcRgP9KnJyH5K0vUrHn9c0ulm7QDoSpOwvybpRtuftP1RSV+UdKCdtgC0bejD+Ih43/ZDkn4saULSMxHxdmudAWjV0KfehtoZ79mBznXypRoAVw7CDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSGOmlpJHPjh07Smuzs7ONtn3vvfdW1ufm5hptf61hZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJDjPjkp79uyprG/btq2yvmnTpjbb+YC63tatW1dam5mZabudscfIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcJ4dlU6cOFFZn5iYGHrbBw8eHHpdSdq1a1dlfcuWLY22v9Y0Crvtk5LOS1qS9H5EbG2jKQDta2Nk/8uIONvCdgB0iPfsQBJNwx6SfmL7iO2pQU+wPWV73vZ8w30BaKDpYfydEXHa9iZJh2z/V0S8vPIJETEtaVqSbEfD/QEYUqORPSJOF7dnJO2XVP0TKAC9GTrstq+1ve7ifUmfk3S0rcYAtKvJYfxmSfttX9zOsxHxo1a6wtgY5999Ly0tVdavuqp8LKv7Hf6rr746VE/jbOiwR8S7kv6sxV4AdIhTb0AShB1IgrADSRB2IAnCDiThiNF9qY1v0GGU9u7dW1q77bbbKte977772m5nZCLCg5YzsgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASTNmMSnv27Kms103pPK6Xoi4ugZ4KIzuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJMF5dlSqm9r4rrvuqqxv2bKltHb+/PnKdaenpyvrdeu/9NJLpbXjx49XrrsW1Y7stp+xfcb20RXLNtg+ZPud4nZ9t20CaGo1h/HfkXT3JcsekXQ4Im6UdLh4DGCM1YY9Il6WdO6SxTsl7Svu75N0f8t9AWjZsO/ZN0fEgiRFxILtTWVPtD0laWrI/QBoSecf0EXEtKRpiYkdgT4Ne+pt0fakJBW3Z9prCUAXhg37AUm7i/u7Jb3QTjsAulI7P7vtGUnbJW2UtChpr6R/l/QDSX8k6VeSvhARl36IN2hbHMZfYep+z/7AAw9U1m+66abS2unTpyvX3bp1a2V9YWGhsl5l8+bNlfXbb7+9sj43Nzf0vrtWNj977Xv2iNhVUvpMo44AjBRflwWSIOxAEoQdSIKwA0kQdiAJfuKKSo8//nhl/b333qusP/zww6W1xcXFynUvXLhQWW+i7tTa7OxsZf1KvBQ1IzuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJFH7E9dWd8ZPXNecdevWDV1fWlqqXPfs2bOV9br1q+zYsaOyfiWfZy/7iSsjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwXl2pJTxUtKM7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBNeNx5q1ffv20trk5GTlujMzMy1307/akd32M7bP2D66Ytmjtn9j+43i755u2wTQ1GoO478j6e4By/81Im4p/n7YblsA2lYb9oh4WdK5EfQCoENNPqB7yPabxWH++rIn2Z6yPW97vsG+ADQ0bNi/JelTkm6RtCDpG2VPjIjpiNgaEVuH3BeAFgwV9ohYjIiliLgg6duStrXbFoC2DRV22yvPW3xe0tGy5wIYD7Xn2W3PSNouaaPtU5L2Stpu+xZJIemkpC932CMw0LZt1QeUU1NTpbW6692vxfPstWGPiF0DFj/dQS8AOsTXZYEkCDuQBGEHkiDsQBKEHUiCn7i2YGJiorK+cePGRutjsCeeeKKyfscdd5TW9u/f33Y7Y4+RHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeS4Dx7C+rOox88eLCyXjd9MAbbsGFDZf2pp54qrT355JNttzP2GNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IIk1c559dna2t31fffXVlfWbb765sn7NNde02c5I1V1y+dlnnx1RJ5c7fvx4ae3cuXzTFzKyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASjojR7czubGd1/x1154Orzsmi3IsvvtiojvZFhActrx3ZbV9v+6e2j9l+2/ZXiuUbbB+y/U5xu77tpgG0ZzWH8e9L+vuIuFnSHZIetP0nkh6RdDgibpR0uHgMYEzVhj0iFiLi9eL+eUnHJF0naaekfcXT9km6v6smATT3ob4bb/sGSbdK+rmkzRGxIC3/g2B7U8k6U5KmmrUJoKlVh932xyQ9J+mrEfFbe+BnAJeJiGlJ08U2RvdpIIAPWNWpN9sf0XLQvxcRzxeLF21PFvVJSWe6aRFAG2pHdi8P4U9LOhYR31xROiBpt6SvF7cvdNLhKs3NzVXWX3nllcr6oUOHKuucmsOVbjWH8XdK+htJb9l+o1j2NS2H/Ae2vyTpV5K+0E2LANpQG/aI+A9JZW/QP9NuOwC6wtdlgSQIO5AEYQeSIOxAEoQdSGLN/MS1zoEDByrrR44cqaw/9thjbbYDdGbon7gCWBsIO5AEYQeSIOxAEoQdSIKwA0kQdiCJNOfZgSw4zw4kR9iBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ1Ibd9vW2f2r7mO23bX+lWP6o7d/YfqP4u6f7dgEMq/biFbYnJU1GxOu210k6Iul+SX8t6XcR8S+r3hkXrwA6V3bxitXMz74gaaG4f972MUnXtdsegK59qPfstm+QdKuknxeLHrL9pu1nbK8vWWfK9rzt+UadAmhk1degs/0xSS9J+qeIeN72ZklnJYWkx7V8qP93NdvgMB7oWNlh/KrCbvsjkuYk/TgivjmgfoOkuYj405rtEHagY0NfcNK2JT0t6djKoBcf3F30eUlHmzYJoDur+TT+05J+JuktSReKxV+TtEvSLVo+jD8p6cvFh3lV22JkBzrW6DC+LYQd6B7XjQeSI+xAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRRe8HJlp2V9D8rHm8slo2jce1tXPuS6G1Ybfb2ibLCSH/PftnO7fmI2NpbAxXGtbdx7Uuit2GNqjcO44EkCDuQRN9hn+55/1XGtbdx7Uuit2GNpLde37MDGJ2+R3YAI0LYgSR6Cbvtu23/0vYJ24/00UMZ2ydtv1VMQ93r/HTFHHpnbB9dsWyD7UO23yluB86x11NvYzGNd8U0472+dn1Pfz7y9+y2JyQdl/RZSackvSZpV0T8YqSNlLB9UtLWiOj9Cxi2/0LS7yR99+LUWrb/WdK5iPh68Q/l+oj4hzHp7VF9yGm8O+qtbJrxv1WPr12b058Po4+RfZukExHxbkT8XtL3Je3soY+xFxEvSzp3yeKdkvYV9/dp+X+WkSvpbSxExEJEvF7cPy/p4jTjvb52FX2NRB9hv07Sr1c8PqXxmu89JP3E9hHbU303M8Dmi9NsFbebeu7nUrXTeI/SJdOMj81rN8z05031EfZBU9OM0/m/OyPizyX9laQHi8NVrM63JH1Ky3MALkj6Rp/NFNOMPyfpqxHx2z57WWlAXyN53foI+ylJ1694/HFJp3voY6CIOF3cnpG0X8tvO8bJ4sUZdIvbMz338/8iYjEiliLigqRvq8fXrphm/DlJ34uI54vFvb92g/oa1evWR9hfk3Sj7U/a/qikL0o60EMfl7F9bfHBiWxfK+lzGr+pqA9I2l3c3y3phR57+YBxmca7bJpx9fza9T79eUSM/E/SPVr+RP6/Jf1jHz2U9PXHkv6z+Hu7794kzWj5sO5/tXxE9CVJfyjpsKR3itsNY9Tbv2l5au83tRysyZ56+7SW3xq+KemN4u+evl+7ir5G8rrxdVkgCb5BByRB2IEkCDuQBGEHkiDsQBKEHUiCsANJ/B9R8fxNdh9SgwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMZ0lEQVR4nO3dYYgc9RnH8d+vNr7RBJKKR7haL61CLIXaEkLBUix6xdybqKCYFyWmwvWFgkJfVOwLhSJIaesLXwSuaJrWVBE0TZCKXoLU9k3wFDXRs2rNtSY575C8iH3Vqk9f7KSc8Xb23JnZ2d7z/cCxu/Ps7Dys/vKf3dmZvyNCAFa/L7TdAIDBIOxAEoQdSIKwA0kQdiCJLw5yY7b56h9oWER4ueWVRnbb19n+m+13bN9d5bUANMv9Hme3fZ6ktySNSzoh6UVJOyLijZJ1GNmBhjUxsm+V9E5EvBsR/5b0uKTtFV4PQIOqhH1U0ntLHp8oln2K7UnbM7ZnKmwLQEVVvqBbblfhM7vpETElaUpiNx5oU5WR/YSkS5Y8/rKkU9XaAdCUKmF/UdLltjfZPl/SLZIO1tMWgLr1vRsfER/ZvkPSs5LOk/RIRLxeW2cAatX3obe+NsZndqBxjfyoBsD/D8IOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkhjolM1oxpo1a7rWRkc/MyPXp+zZs6e0vmvXrtL63NxcaR3Dg5EdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgOPsqcOmll3atHTx4sHTdI0eOlNaPHz9eWh8fHy+tHzp0qLSOwakUdttzkj6U9LGkjyJiSx1NAahfHSP79yPigxpeB0CD+MwOJFE17CHpOdsv2Z5c7gm2J23P2J6puC0AFVTdjb8qIk7ZvljStO03I+KFpU+IiClJU5JkOypuD0CfKo3sEXGquF2UtF/S1jqaAlC/vsNu+wLba8/el/QDScfqagxAvRzR35617a+qM5pLnY8Df4iI+3usw258A9atW9e1dsUVV5Su2+s4+6233lpav/nmm0vrExMTpXXULyK83PK+P7NHxLuSvtl3RwAGikNvQBKEHUiCsANJEHYgCcIOJNH3obe+Nsaht1VnZGSktL6wsDCgTnBWt0NvjOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kASXkl4FyqZsXrt2bem6p0+fLq2PjY2V1nudQnvZZZd1rT300EOl66JejOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATH2VeBsimbd+/eXbruvn37Sut79uwprXOp6P8fjOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATH2VeBxcXFrrX333+/dN1ex9F37drVV08YPj1HdtuP2F60fWzJsg22p22/Xdyub7ZNAFWtZDf+t5KuO2fZ3ZIOR8Tlkg4XjwEMsZ5hj4gXJJ177aLtkvYW9/dKur7mvgDUrN/P7CMRMS9JETFv++JuT7Q9KWmyz+0AqEnjX9BFxJSkKYmJHYE29XvobcH2Rkkqbrt/HQxgKPQb9oOSdhb3d0o6UE87AJrSc352249JulrSRZIWJN0r6Y+SnpD0FUn/lHRTRJRfgFzsxrdh3bp1pfUNGzaU1ufm5krr27ZtK61z3fjB6zY/e8/P7BGxo0vpmkodARgofi4LJEHYgSQIO5AEYQeSIOxAEpziusqdOXOmUh2rByM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBcfZVoOw01htvvLF03enp6dL6yZMnS+ubN28urR89erS0jsFhZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJDjOvspdc035RYAXFhZK672Os4+Pj5fWH3zwwdI6BoeRHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeS4Dj7KnDgwIGute3bt5euOzU1VVp/5pln+uoJw6fnyG77EduLto8tWXaf7ZO2Xyn+JpptE0BVK9mN/62k65ZZ/mBEXFn8/anetgDUrWfYI+IFSacH0AuABlX5gu4O268Vu/nruz3J9qTtGdszFbYFoKJ+w75b0tckXSlpXtKvuj0xIqYiYktEbOlzWwBq0FfYI2IhIj6OiE8k/UbS1nrbAlC3vsJue+OShzdIOtbtuQCGQ8/j7LYfk3S1pItsn5B0r6SrbV8pKSTNSfpxgz2ih9HR0a61XvOvl11zXpLGxsZK673Oh8fw6Bn2iNixzOKHG+gFQIP4uSyQBGEHkiDsQBKEHUiCsANJcIrrKrB///7GXvv48eOldduNbRv1YmQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQcEYPbmD24jSUyMjLStXbttdeWrtvrFNXp6enSOsfZh09ELPsfhZEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgfPZVYO3atV1rW7eWz9/R61LSWD0Y2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCc5nR6le142fnZ0trU9MTNTZDlag7/PZbV9i+3nbs7Zft31nsXyD7Wnbbxe36+tuGkB9VrIb/5Gkn0TEFZK+I+l221+XdLekwxFxuaTDxWMAQ6pn2CNiPiJeLu5/KGlW0qik7ZL2Fk/bK+n6ppoEUN3n+m287TFJ35J0RNJIRMxLnX8QbF/cZZ1JSZPV2gRQ1YrDbvtCSU9Kuisizqz0QoMRMSVpqngNvqADWrKiQ2+216gT9H0R8VSxeMH2xqK+UdJiMy0CqEPPkd2dIfxhSbMR8eslpYOSdkp6oLg90EiHaNWmTZtK66+++mppffPmzV1rb775Zl89oT8r2Y2/StIPJR21/Uqx7B51Qv6E7dsk/VPSTc20CKAOPcMeEX+V1O0D+jX1tgOgKfxcFkiCsANJEHYgCcIOJEHYgSS4lDQqOXToUGl9fHy8a43j7IPFyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSXApaVSybdu20vqjjz7atXbmzJnSdXudS4/l9X0paQCrA2EHkiDsQBKEHUiCsANJEHYgCcIOJMFxdmCV4Tg7kBxhB5Ig7EAShB1IgrADSRB2IAnCDiTRM+y2L7H9vO1Z26/bvrNYfp/tk7ZfKf4mmm8XQL96/qjG9kZJGyPiZdtrJb0k6XpJN0v6V0T8csUb40c1QOO6/ahmJfOzz0uaL+5/aHtW0mi97QFo2uf6zG57TNK3JB0pFt1h+zXbj9he32WdSdsztmcqdQqgkhX/Nt72hZL+LOn+iHjK9oikDySFpJ+rs6v/ox6vwW480LBuu/ErCrvtNZKelvRsRPx6mfqYpKcj4hs9XoewAw3r+0QY25b0sKTZpUEvvrg76wZJx6o2CaA5K/k2/ruS/iLpqKRPisX3SNoh6Up1duPnJP24+DKv7LUY2YGGVdqNrwthB5rH+exAcoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkel5wsmYfSPrHkscXFcuG0bD2Nqx9SfTWrzp7u7RbYaDns39m4/ZMRGxprYESw9rbsPYl0Vu/BtUbu/FAEoQdSKLtsE+1vP0yw9rbsPYl0Vu/BtJbq5/ZAQxO2yM7gAEh7EASrYTd9nW2/2b7Hdt3t9FDN7bnbB8tpqFudX66Yg69RdvHlizbYHva9tvF7bJz7LXU21BM410yzXir713b058P/DO77fMkvSVpXNIJSS9K2hERbwy0kS5sz0naEhGt/wDD9vck/UvS785OrWX7F5JOR8QDxT+U6yPip0PS2336nNN4N9Rbt2nGb1WL712d05/3o42RfaukdyLi3Yj4t6THJW1voY+hFxEvSDp9zuLtkvYW9/eq8z/LwHXpbShExHxEvFzc/1DS2WnGW33vSvoaiDbCPirpvSWPT2i45nsPSc/Zfsn2ZNvNLGPk7DRbxe3FLfdzrp7TeA/SOdOMD81718/051W1EfblpqYZpuN/V0XEtyVtk3R7sbuKldkt6WvqzAE4L+lXbTZTTDP+pKS7IuJMm70stUxfA3nf2gj7CUmXLHn8ZUmnWuhjWRFxqrhdlLRfnY8dw2Th7Ay6xe1iy/38T0QsRMTHEfGJpN+oxfeumGb8SUn7IuKpYnHr791yfQ3qfWsj7C9Kutz2JtvnS7pF0sEW+vgM2xcUX5zI9gWSfqDhm4r6oKSdxf2dkg602MunDMs03t2mGVfL713r059HxMD/JE2o84383yX9rI0euvT1VUmvFn+vt92bpMfU2a37jzp7RLdJ+pKkw5LeLm43DFFvv1dnau/X1AnWxpZ6+646Hw1fk/RK8TfR9ntX0tdA3jd+LgskwS/ogCQIO5AEYQeSIOxAEoQdSIKwA0kQdiCJ/wLjzOUpOUGBhgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMyklEQVR4nO3db4hddX7H8c+ndvPAbDBjJSF1paaLypZCkyWGwi7FsCRYQeIils2DEllh9kGUjVRc2T6IUgpS6vaBDxYTHHZaVpeFGBLW0kTCUq0PguM41Ziw0cZ0dzbDDCGRJA/CNsm3D+ZMO8a5587cc8491/m+X3C5957vnHO+XP3knHvPn58jQgCWv99ruwEA/UHYgSQIO5AEYQeSIOxAEr/fz5XZ5qd/oGER4YWmV9qy277f9q9sf2z7mSrLAtAs93qc3fZNkk5J2ippUtI7knZExImSediyAw1rYsu+WdLHEXE6In4n6WeStldYHoAGVQn77ZJ+M+/9ZDHtM2wP2x6zPVZhXQAqqvID3UK7Cp/bTY+IvZL2SuzGA22qsmWflHTHvPdfkXS2WjsAmlIl7O9Iusv2etsrJH1H0qF62gJQt5534yPiqu3HJR2WdJOkkYj4sLbOANSq50NvPa2M7+xA4xo5qQbAFwdhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0n0dchmYCkuXLhQWh8aGupTJ8sDW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0lUOqnG9hlJlyRdk3Q1IjbV0RSA+tVxBt2WiDhXw3IANIjdeCCJqmEPSUdsv2t7eKE/sD1se8z2WMV1AajAEdH7zPYfRsRZ22skvSHpiYh4s+Tve18Z0uFCmN5EhBeaXmnLHhFni+cZSQckba6yPADN6TnstlfaXjX3WtI2ScfragxAvar8Gr9W0gHbc8t5JSL+rZausCSPPvpox9r58+dL5z106FDN3dTnlVdeabuFZaXnsEfEaUl/VmMvABrEoTcgCcIOJEHYgSQIO5AEYQeSqHQG3ZJXxhl0jSj7bzg9PV0678jISGn97bffLq2//vrrpfUyTzzxRGl93759pfUrV670vO7lrJEz6AB8cRB2IAnCDiRB2IEkCDuQBGEHkiDsQBIcZ18GLl261LFWdvmrJG3eXH6/kRUrVpTWn3zyydJ6mffee6+0vmXLltL6p59+2vO6lzOOswPJEXYgCcIOJEHYgSQIO5AEYQeSIOxAEnUM7IiWXb16tWPt+PHyW/m/8MILpfUDBw701BMGD1t2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC4+zL3KlTp0rrd999d2n9+vXrdbaDFnXdstsesT1j+/i8abfafsP2R8XzULNtAqhqMbvxP5F0/w3TnpF0NCLuknS0eA9ggHUNe0S8Ken8DZO3SxotXo9KeqjmvgDUrNfv7GsjYkqSImLK9ppOf2h7WNJwj+sBUJPGf6CLiL2S9krccBJoU6+H3qZtr5Ok4nmmvpYANKHXsB+StLN4vVPSwXraAdCUrrvxtl+VdJ+k22xPStoj6XlJP7f9mKRfS3qkySZRbv369R1r4+PjpfNu3Lix7nY+Y82ajj/n6Ny5c6Xzcoy/Xl3DHhE7OpS+VXMvABrE6bJAEoQdSIKwA0kQdiAJwg4kwSWuy0CVW0lv2LChtD4xMVFav+eee0rrIyMjHWu7du0qnffixYuldSwNW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILj7MvA5cuXO9a6Dck8OjpaWt+3b19p/eGHHy6tnz9/4+0L/1+3S1xRL7bsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEx9mXucnJydL6sWPHSusvvvhiaX3//v2l9aeffrpjrVtvqBdbdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IguPsy9wtt9xSWr/33nsrLf+tt94qrZ8+fbrS8lGfrlt22yO2Z2wfnzftWdu/tT1RPB5otk0AVS1mN/4nku5fYPo/RcSG4vGv9bYFoG5dwx4Rb0rqfG8hAF8IVX6ge9z2+8Vu/lCnP7I9bHvM9liFdQGoqNew/1jSVyVtkDQlqeNdDSNib0RsiohNPa4LQA16CntETEfEtYi4LmmfpM31tgWgbj2F3fa6eW+/Lal8XGAAret6nN32q5Luk3Sb7UlJeyTdZ3uDpJB0RtL3GuwRFaxataq0PjMzU1pfv359ab3bGOvbtm3rWDty5EjpvKhX17BHxI4FJr/cQC8AGsTpskAShB1IgrADSRB2IAnCDiTBJa7LwOrVqzvWxsfHS+ftdivoM2fOlNZtl9ZXrlzZ87wRUVrH0rBlB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkOM6+DFy7dq1jrdtx9EceeaTSup966qnS+oEDBzrWPvnkk9J5JyYmeuoJC2PLDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcJx9GdiyZUvHWtXj6Fg+2LIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBIcZ18GRkdHO9aGhob62MnnlV1PPz093cdO0HXLbvsO27+0fdL2h7a/X0y/1fYbtj8qntv9vwpAqcXsxl+V9DcR8TVJfy5pl+0/kfSMpKMRcZeko8V7AAOqa9gjYioixovXlySdlHS7pO2S5vYfRyU91FSTAKpb0nd223dK2ijpmKS1ETElzf6DYHtNh3mGJQ1XaxNAVYsOu+0vS9ovaXdEXOw2KN+ciNgraW+xDEbqA1qyqENvtr+k2aD/NCJeKyZP215X1NdJmmmmRQB16Lpl9+wm/GVJJyPiR/NKhyTtlPR88XywkQ7R1c0339yx9txzz5XOu2fPnkrr3rFjR2n9xIkTHWtTU1OV1o2lWcxu/Dck/bWkD2zP3cj7h5oN+c9tPybp15K4cBoYYF3DHhH/IanTF/Rv1dsOgKZwuiyQBGEHkiDsQBKEHUiCsANJcInrMvDggw92rHUbsnnr1q2V1n3wYPnpFYcPH660fNSHLTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJOGI/t08hjvVNGP16tUdaxcuXKi07Jdeeqm0vnv37tL6lStXKq0fSxcRC16lypYdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LgODuwzHCcHUiOsANJEHYgCcIOJEHYgSQIO5AEYQeS6Bp223fY/qXtk7Y/tP39Yvqztn9re6J4PNB8uwB61fWkGtvrJK2LiHHbqyS9K+khSX8l6XJE/OOiV8ZJNUDjOp1Us5jx2ackTRWvL9k+Ken2etsD0LQlfWe3faekjZKOFZMet/2+7RHbQx3mGbY9ZnusUqcAKln0ufG2vyzp3yX9fUS8ZnutpHOSQtLfaXZX/7tdlsFuPNCwTrvxiwq77S9J+oWkwxHxowXqd0r6RUT8aZflEHagYT1fCGPbkl6WdHJ+0Isf7uZ8W9Lxqk0CaM5ifo3/pqS3JH0g6Xox+YeSdkjaoNnd+DOSvlf8mFe2LLbsQMMq7cbXhbADzeN6diA5wg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBJdbzhZs3OS/nve+9uKaYNoUHsb1L4keutVnb39UadCX69n/9zK7bGI2NRaAyUGtbdB7Uuit171qzd244EkCDuQRNth39vy+ssMam+D2pdEb73qS2+tfmcH0D9tb9kB9AlhB5JoJey277f9K9sf236mjR46sX3G9gfFMNStjk9XjKE3Y/v4vGm32n7D9kfF84Jj7LXU20AM410yzHirn13bw5/3/Tu77ZsknZK0VdKkpHck7YiIE31tpAPbZyRtiojWT8Cw/ReSLkv657mhtWz/g6TzEfF88Q/lUET8YEB6e1ZLHMa7od46DTP+qFr87Ooc/rwXbWzZN0v6OCJOR8TvJP1M0vYW+hh4EfGmpPM3TN4uabR4ParZ/1n6rkNvAyEipiJivHh9SdLcMOOtfnYlffVFG2G/XdJv5r2f1GCN9x6Sjth+1/Zw280sYO3cMFvF85qW+7lR12G8++mGYcYH5rPrZfjzqtoI+0JD0wzS8b9vRMTXJf2lpF3F7ioW58eSvqrZMQCnJL3QZjPFMOP7Je2OiItt9jLfAn315XNrI+yTku6Y9/4rks620MeCIuJs8Twj6YBmv3YMkum5EXSL55mW+/k/ETEdEdci4rqkfWrxsyuGGd8v6acR8VoxufXPbqG++vW5tRH2dyTdZXu97RWSviPpUAt9fI7tlcUPJ7K9UtI2Dd5Q1Ick7Sxe75R0sMVePmNQhvHuNMy4Wv7sWh/+PCL6/pD0gGZ/kf8vSX/bRg8d+vpjSf9ZPD5suzdJr2p2t+5/NLtH9JikP5B0VNJHxfOtA9Tbv2h2aO/3NRusdS319k3NfjV8X9JE8Xig7c+upK++fG6cLgskwRl0QBKEHUiCsANJEHYgCcIOJEHYgSQIO5DE/wKpWQlgDY1hjQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAALYUlEQVR4nO3dQaxcZ3nG8f/TAJsQqU6jWCaEhlbZsQhVlE2jKixAaTYOCyqyMqLSZdFUdEdEF0RCSKhq6bKSEREG0SCkJI0VVYUoQoQVihOliYMFSZEBY8tW5FYNK0rydnGPoxvn3jvXc2bmzL3v/yeNZubcmXPeGfu53/edb+Z+qSokHXx/MHUBklbDsEtNGHapCcMuNWHYpSbes8qDJfHUv7RkVZXtto9q2ZPcm+RnSV5L8tCYfUlarsw7z57kOuDnwMeBc8BzwANV9dNdnmPLLi3ZMlr2u4DXquoXVfU74LvA0RH7k7REY8J+C/DrLffPDdveIclGklNJTo04lqSRxpyg266r8K5uelUdB46D3XhpSmNa9nPArVvufxA4P64cScsyJuzPAbcn+XCS9wGfBk4upixJizZ3N76qfp/kQeD7wHXAI1X1ysIqk7RQc0+9zXUwx+zS0i3lQzWS9g/DLjVh2KUmDLvUhGGXmjDsUhOGXWrCsEtNGHapCcMuNWHYpSYMu9SEYZeaMOxSE4ZdasKwS00YdqkJwy41YdilJgy71IRhl5ow7FIThl1qwrBLTRh2qQnDLjVh2KUmDLvUhGGXmph7yWZparNWIE62Xcy0rVFhT3IWeAN4E/h9Vd25iKIkLd4iWvaPVdXrC9iPpCVyzC41MTbsBfwgyfNJNrZ7QJKNJKeSnBp5LEkjZNZJjl2fnHygqs4nuRl4Gvjbqnp2l8fPfzDpKp6g215VbfvCR7XsVXV+uL4EPAHcNWZ/kpZn7rAnuT7JDVduA58ATi+qMEmLNeZs/GHgiaGr9B7gX6vqPxZSlbQAu3XzO3bxR43Zr/lgjtm1QCPPNy2wkvWylDG7pP3DsEtNGHapCcMuNWHYpSb8iusCLHtG4yCfOdbq2LJLTRh2qQnDLjVh2KUmDLvUhGGXmjDsUhPOs6/ArHnyWfP0Y/4iy0H+ay5j39dubNmlJgy71IRhl5ow7FIThl1qwrBLTRh2qQnn2dfA2LnugzqffJA/IzAFW3apCcMuNWHYpSYMu9SEYZeaMOxSE4ZdasJ59n3goM6ja7VmtuxJHklyKcnpLdtuTPJ0kleH60PLLVPSWHvpxn8TuPeqbQ8Bz1TV7cAzw31Ja2xm2KvqWeDyVZuPAieG2yeA+xdcl6QFm3fMfriqLgBU1YUkN+/0wCQbwMacx5G0IEs/QVdVx4HjAEk80yRNZN6pt4tJjgAM15cWV5KkZZg37CeBY8PtY8CTiylH0rJkD98ZfhS4B7gJuAh8Cfg34HvAh4BfAZ+qqqtP4m23rwPZjZ96Hvyg/t34sbXv9vx1ft1jVdW2L25m2BfJsC+HYb/256/z6x5rp7D7cVmpCcMuNWHYpSYMu9SEYZea8CuuC7DOZ3bHLmu8zq9N18aWXWrCsEtNGHapCcMuNWHYpSYMu9SEYZeaMOxSE4ZdasKwS00YdqkJwy41YdilJgy71IRhl5ow7FIThl1qwrBLTRh2qQnDLjVh2KUmDLvUhGGXmjDsUhMzw57kkSSXkpzesu3hJL9J8uJwuW+5ZUoaay8t+zeBe7fZ/s9Vdcdw+ffFliVp0WaGvaqeBS6voBZJSzRmzP5gkpeGbv6hnR6UZCPJqSSnRhxL0kiZtbAfQJLbgKeq6iPD/cPA60ABXwaOVNVn97Cf2QfTSq3zwo5ja9vt+Qd5wcqq2vbFzdWyV9XFqnqzqt4Cvg7cNaY4Scs3V9iTHNly95PA6Z0eK2k9zFyfPcmjwD3ATUnOAV8C7klyB5vd+LPA55ZYY3t7GWrp3Q5yV30eexqzL+xgjtnnMmXY9/OYvauFjtkl7T+GXWrCsEtNGHapCcMuNTFz6k3LN/Zs+25npcfue53PiK9zbevIll1qwrBLTRh2qQnDLjVh2KUmDLvUhGGXmnCefR9Y5nzxmL/2spefT1m73smWXWrCsEtNGHapCcMuNWHYpSYMu9SEYZeacJ5du1r2PLxWx5ZdasKwS00YdqkJwy41YdilJgy71IRhl5pwnn0N7OfvZS+zdufoF2tmy57k1iQ/THImyStJPj9svzHJ00leHa4PLb9cSfOauT57kiPAkap6IckNwPPA/cBngMtV9dUkDwGHquoLM/blr+oV28+rpuzn2qc09/rsVXWhql4Ybr8BnAFuAY4CJ4aHnWDzF4CkNXVNY/YktwEfBX4CHK6qC7D5CyHJzTs8ZwPYGFempLFmduPffmDyfuBHwFeq6vEk/1NVf7jl5/9dVbuO2+3Gr95+7grv59qnNHc3HiDJe4HHgO9U1ePD5ovDeP7KuP7SIgqVtBx7ORsf4BvAmar62pYfnQSODbePAU8uvjztRVXteEmy60V97OVs/N3Aj4GXgbeGzV9kc9z+PeBDwK+AT1XV5Rn7shu/BLv9G+7nQNuNn89O3fg9j9kXwbAvh2HXVqPG7JL2P8MuNWHYpSYMu9SEYZea8Cuu+4Bf9dQi2LJLTRh2qQnDLjVh2KUmDLvUhGGXmjDsUhPOs6+BsfPofvtLe2HLLjVh2KUmDLvUhGGXmjDsUhOGXWrCsEtNOM++Bpwn1yrYsktNGHapCcMuNWHYpSYMu9SEYZeaMOxSE3tZn/3WJD9McibJK0k+P2x/OMlvkrw4XO5bfrnqxLXlF2sv67MfAY5U1QtJbgCeB+4H/gr4bVX9454P5pLN0tLttGTzzE/QVdUF4MJw+40kZ4BbFluepGW7pjF7ktuAjwI/GTY9mOSlJI8kObTDczaSnEpyalSlkkaZ2Y1/+4HJ+4EfAV+pqseTHAZeBwr4Mptd/c/O2IfdeGnJdurG7ynsSd4LPAV8v6q+ts3PbwOeqqqPzNiPYZeWbKew7+VsfIBvAGe2Bn04cXfFJ4HTY4uUtDx7ORt/N/Bj4GXgrWHzF4EHgDvY7MafBT43nMzbbV+27NKSjerGL4phl5Zv7m68pIPBsEtNGHapCcMuNWHYpSYMu9SEYZeaMOxSE4ZdasKwS00YdqkJwy41YdilJgy71MSql2x+Hfjllvs3DdvW0brWtq51gbXNa5G1/fFOP1jp99nfdfDkVFXdOVkBu1jX2ta1LrC2ea2qNrvxUhOGXWpi6rAfn/j4u1nX2ta1LrC2ea2ktknH7JJWZ+qWXdKKGHapiUnCnuTeJD9L8lqSh6aoYSdJziZ5eViGetL16YY19C4lOb1l241Jnk7y6nC97Rp7E9W2Fst477LM+KTv3dTLn698zJ7kOuDnwMeBc8BzwANV9dOVFrKDJGeBO6tq8g9gJPkL4LfAt64srZXkH4DLVfXV4Rfloar6wprU9jDXuIz3kmrbaZnxzzDhe7fI5c/nMUXLfhfwWlX9oqp+B3wXODpBHWuvqp4FLl+1+ShwYrh9gs3/LCu3Q21roaouVNULw+03gCvLjE/63u1S10pMEfZbgF9vuX+O9VrvvYAfJHk+ycbUxWzj8JVltobrmyeu52ozl/FepauWGV+b926e5c/HmiLs2y1Ns07zf39eVX8G/CXwN0N3VXvzL8CfsrkG4AXgn6YsZlhm/DHg76rqf6esZatt6lrJ+zZF2M8Bt265/0Hg/AR1bKuqzg/Xl4An2Bx2rJOLV1bQHa4vTVzP26rqYlW9WVVvAV9nwvduWGb8MeA7VfX4sHny9267ulb1vk0R9ueA25N8OMn7gE8DJyeo412SXD+cOCHJ9cAnWL+lqE8Cx4bbx4AnJ6zlHdZlGe+dlhln4vdu8uXPq2rlF+A+Ns/I/xfw91PUsENdfwL853B5ZeragEfZ7Nb9H5s9or8G/gh4Bnh1uL5xjWr7NptLe7/EZrCOTFTb3WwODV8CXhwu90393u1S10reNz8uKzXhJ+ikJgy71IRhl5ow7FIThl1qwrBLTRh2qYn/B5K6DwsWf6HqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMOklEQVR4nO3dX4hc9RnG8edpqggqmFQSY4yNjSItBWMJWlCKRZQ0XkQvLOaiJlTZJSgaKNhgLyLUgrS1vVRWFLfFGiUqBo3/CEHrhZpV0hhN1VVSjVmzpLnwH2iNby/2RNY4c2Yz58ycMe/3A8PMnHdmzsvok9+Z+c2enyNCAI5+32m6AQD9QdiBJAg7kARhB5Ig7EAS3+3nzmzz1T/QYxHhVtsrjey2l9l+w/a47XVVXgtAb7nbeXbbsyS9KekSSXskbZO0MiJeL3kOIzvQY70Y2c+TNB4R70TE55I2SFpR4fUA9FCVsC+Q9N60+3uKbV9je8j2mO2xCvsCUFGVL+haHSp84zA9IkYkjUgcxgNNqjKy75G0cNr90yTtrdYOgF6pEvZtks6yfYbtYyVdJWlTPW0BqFvXh/ER8YXt6yU9JWmWpHsi4rXaOgNQq66n3rraGZ/ZgZ7ryY9qAHx7EHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kERfTyWNfNavX9+2dsopp5Q+d82aNXW3kxojO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwTw7So2OjpbWt27d2qdOUBUjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwTz7Ue7ZZ58trZ9++uml9blz55bWL7vsstL6rFmz2tY2bNhQ+lzUq1LYbe+W9JGkg5K+iIildTQFoH51jOw/j4j9NbwOgB7iMzuQRNWwh6Snbb9se6jVA2wP2R6zPVZxXwAqqHoYf0FE7LU9V9Iztv8dEc9Nf0BEjEgakSTbUXF/ALpUaWSPiL3F9aSkRySdV0dTAOrXddhtH2/7xEO3JV0qaWddjQGoV5XD+HmSHrF96HX+ERFP1tIVarNgwYLS+qJFi0rra9euLa1v2bLlSFv6yoEDB7p+Lo5c12GPiHcknVNjLwB6iKk3IAnCDiRB2IEkCDuQBGEHkuBPXJMbHh4urW/cuLG0zvTZtwcjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwTx7cjt3lp+CgHn0owcjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkOobd9j22J23vnLZtju1nbL9VXM/ubZsAqprJyH6vpGWHbVsnaUtEnCVpS3EfwADrGPaIeE7S4ecmWiFptLg9KunymvsCULNuz0E3LyImJCkiJmzPbfdA20OShrrcD4Ca9PyEkxExImlEkmxHr/cHoLVuv43fZ3u+JBXXk/W1BKAXug37JkmriturJD1aTzsAeqXjYbzt+yVdJOlk23skrZd0m6QHbV8j6V1JV/aySTTn2muvLa2fc845pfXNmze3rT3xxBNd9YTudAx7RKxsU7q45l4A9BC/oAOSIOxAEoQdSIKwA0kQdiAJlmxObnh4uLR+/vnnl9bPPvvs0vr+/fvb1ph66y9GdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1Ignn25K6++upKz3/yySdL688//3yl10d9GNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnm2ZPbuHFjaf2DDz4orT/wwAOldebZBwcjO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4k4Yjo387s/u0skdWrV7etdTrv++23315aHx8f76YlNCgi3Gp7x5Hd9j22J23vnLbtFtvv295eXJbX2SyA+s3kMP5eSctabP9rRCwpLpvrbQtA3TqGPSKek3SgD70A6KEqX9Bdb3tHcZg/u92DbA/ZHrM9VmFfACrqNux3SFosaYmkCUltv+WJiJGIWBoRS7vcF4AadBX2iNgXEQcj4ktJd0k6r962ANStq7Dbnj/t7hWSdrZ7LIDB0HGe3fb9ki6SdLKkfZLWF/eXSApJuyUNR8REx50xz94TZf8NzzzzzNLnvv3223W3g4a1m2fvePKKiFjZYvPdlTsC0Ff8XBZIgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSRYshlHreOOO65tbcmSJaXPfeGFF+pup3GM7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBPPsaMycOXNK66eeemql11+4cGHb2ubN5WuR2i3PxvytxsgOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwz34U2L17d9vavHnzSp978ODB0vrk5GRp/dNPPy2tn3TSSW1ra9asKX3urbfeWlr/7LPPSusTE+1XES97z45WHUd22wttb7W9y/Zrtm8sts+x/Yztt4rr2b1vF0C3ZnIY/4Wk30TEDyX9VNJ1tn8kaZ2kLRFxlqQtxX0AA6pj2CNiIiJeKW5/JGmXpAWSVkgaLR42KunyXjUJoLoj+sxue5GkcyW9KGleRExIU/8g2J7b5jlDkoaqtQmgqhmH3fYJkh6StDYiPpzpHwpExIikkeI1opsmAVQ3o6k328doKuj3RcTDxeZ9tucX9fmSyr+2BdAoR5QPtp4awkclHYiItdO2/0nSfyPiNtvrJM2JiJs6vBYje5+Nj4+X1hcvXlxaX7VqVWn98ccfL63fdFP7/yVuuOGG0ud+8sknpfWXXnqptL58+fLS+tEqIloeds/kMP4CSb+S9Krt7cW2myXdJulB29dIelfSlXU0CqA3OoY9Ip6X1O4D+sX1tgOgV/i5LJAEYQeSIOxAEoQdSIKwA0l0nGevdWfMs/dd1Xn2XrrzzjtL653+BBattZtnZ2QHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSSYZweOMsyzA8kRdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBIdw257oe2ttnfZfs32jcX2W2y/b3t7ccm5GDbwLdHx5BW250uaHxGv2D5R0suSLpf0S0kfR8SfZ7wzTl4B9Fy7k1fMZH32CUkTxe2PbO+StKDe9gD02hF9Zre9SNK5kl4sNl1ve4fte2zPbvOcIdtjtscqdQqgkhmfg872CZKelfSHiHjY9jxJ+yWFpN9r6lD/1x1eg8N4oMfaHcbPKOy2j5H0mKSnIuIvLeqLJD0WET/u8DqEHeixrk84aduS7pa0a3rQiy/uDrlC0s6qTQLonZl8G3+hpH9KelXSl8XmmyWtlLREU4fxuyUNF1/mlb0WIzvQY5UO4+tC2IHe47zxQHKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJDqecLJm+yX9Z9r9k4ttg2hQexvUviR661advX2/XaGvf8/+jZ3bYxGxtLEGSgxqb4Pal0Rv3epXbxzGA0kQdiCJpsM+0vD+ywxqb4Pal0Rv3epLb41+ZgfQP02P7AD6hLADSTQSdtvLbL9he9z2uiZ6aMf2btuvFstQN7o+XbGG3qTtndO2zbH9jO23iuuWa+w11NtALONdssx4o+9d08uf9/0zu+1Zkt6UdImkPZK2SVoZEa/3tZE2bO+WtDQiGv8Bhu2fSfpY0t8OLa1l+4+SDkTEbcU/lLMj4rcD0tstOsJlvHvUW7tlxlerwfeuzuXPu9HEyH6epPGIeCciPpe0QdKKBvoYeBHxnKQDh21eIWm0uD2qqf9Z+q5NbwMhIiYi4pXi9keSDi0z3uh7V9JXXzQR9gWS3pt2f48Ga733kPS07ZdtDzXdTAvzDi2zVVzPbbifw3VcxrufDltmfGDeu26WP6+qibC3WppmkOb/LoiIn0j6haTrisNVzMwdkhZrag3ACUm3N9lMscz4Q5LWRsSHTfYyXYu++vK+NRH2PZIWTrt/mqS9DfTRUkTsLa4nJT2iqY8dg2TfoRV0i+vJhvv5SkTsi4iDEfGlpLvU4HtXLDP+kKT7IuLhYnPj712rvvr1vjUR9m2SzrJ9hu1jJV0laVMDfXyD7eOLL05k+3hJl2rwlqLeJGlVcXuVpEcb7OVrBmUZ73bLjKvh967x5c8jou8XScs19Y3825J+10QPbfr6gaR/FZfXmu5N0v2aOqz7n6aOiK6R9D1JWyS9VVzPGaDe/q6ppb13aCpY8xvq7UJNfTTcIWl7cVne9HtX0ldf3jd+LgskwS/ogCQIO5AEYQeSIOxAEoQdSIKwA0kQdiCJ/wPR3toNyBKp3wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAL+klEQVR4nO3dXYhc5R3H8d9PTW9UMKkY1yRUDaG0FKolaEEpFlGsIlGkxSAlRXFzoRBBsWKDFUpBitoLL8T1BdOikeILBik1EmIToUhWMTExahJNmzVLouRCc2XVfy/2pKxx58x6XuZM8v9+YJiZ88yc588hvzxnzss+jggBOP6d0HUBAAaDsANJEHYgCcIOJEHYgSROGmRntjn0D7QsIjzT8loju+0rbL9ne7ftu+qsC0C7XPU8u+0TJb0v6TJJE5K2SFoeEe+UfIeRHWhZGyP7BZJ2R8QHEfG5pGckLauxPgAtqhP2BZL2TXs/USz7Gtujtsdtj9foC0BNdQ7QzbSr8I3d9IgYkzQmsRsPdKnOyD4hadG09wsl7a9XDoC21An7FklLbJ9j+zuSrpe0rpmyADSt8m58RHxh+1ZJL0s6UdITEbGjscoANKryqbdKnfGbHWhdKxfVADh2EHYgCcIOJEHYgSQIO5AEYQeSGOj97Khm3759pe0LFy5sre+bb765tP2xxx5rrW80i5EdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBLe4HgduvPHGnm0ffvhhrXVfffXVpe3Lly8vbV+7dm2t/tEcRnYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILz7MeBLVu29Gzbvn17rXXfcMMNpe2LFi2qtX4MTq2w294r6TNJX0r6IiKWNlEUgOY1MbL/PCI+aWA9AFrEb3YgibphD0nrbb9he3SmD9getT1ue7xmXwBqqLsbf1FE7Ld9hqRXbL8bEZumfyAixiSNSZLtqNkfgIpqjewRsb94PijpBUkXNFEUgOZVDrvtk22feuS1pMsl1TvPA6A1dXbj50t6wfaR9TwdEf9opKpk7r///tL20047rbT9hBM4zor+Koc9Ij6Q9OMGawHQIoYEIAnCDiRB2IEkCDuQBGEHkuAW12PAI488Utr+8ccfD6gSHMsY2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCc6zD4E77rij6xKQACM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgST6ht32E7YP2t4+bdk826/Y3lU8z223TAB1zWZkf1LSFUctu0vShohYImlD8R7AEOsb9ojYJOnQUYuXSVpTvF4j6ZqG6wLQsKp/g25+RExKUkRM2j6j1wdtj0oardgPgIa0/gcnI2JM0pgk2Y62+wMws6pH4w/YHpGk4vlgcyUBaEPVsK+TtKJ4vULSi82UA6Atszn1tlbSvyR93/aE7Zsk3SfpMtu7JF1WvAcwxPr+Zo+I5T2aLm24FgAt4go6IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSaP0v1WC4PfTQQ6XtW7duLW1/+umnmywHLWJkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkOM+e3FlnnVXa3u88+6FDR08DiGHFyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSXCe/Ti3evXq0vbXXnuttH3jxo1NloMOzWZ+9idsH7S9fdqye21/ZPut4nFlu2UCqGs2u/FPSrpihuV/jojzisffmy0LQNP6hj0iNknimkjgGFfnAN2ttrcVu/lze33I9qjtcdvjNfoCUFPVsD8sabGk8yRNSnqg1wcjYiwilkbE0op9AWhApbBHxIGI+DIivpL0qKQLmi0LQNMqhd32yLS310ra3uuzAIZD3/PsttdKukTS6bYnJP1e0iW2z5MUkvZKWtlijejj9ttv79k2Ojpa+t1Vq1aVtu/Zs6dSTRg+fcMeEctnWPx4C7UAaBGXywJJEHYgCcIOJEHYgSQIO5AEt7geA1auLD+zWXbqbfPmzaXf3bVrV6WacOxhZAeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBwRg+vMHlxnx5F9+/aVtm/btq1nW9k5eEl69913K9XUhMWLF5e2j4yMlLb3+zPYWUWEZ1rOyA4kQdiBJAg7kARhB5Ig7EAShB1IgrADSXA/+3Hg2Wef7dl25plnln63X3ubrrrqqtL2Cy+8sLT9nnvuabKcb+XVV1/trO+qGNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnuZz8G9LuffeHCha31feDAgdL2OXPmlLbPmzevct+HDx8ubd+/f3/ldZ90UvklJueee25puz3jLeNDofL97LYX2d5oe6ftHbZXFcvn2X7F9q7ieW7TRQNozmx247+QdHtE/EDSTyXdYvuHku6StCEilkjaULwHMKT6hj0iJiPizeL1Z5J2SlogaZmkNcXH1ki6pq0iAdT3ra6Nt322pPMlvS5pfkRMSlP/Idg+o8d3RiWN1isTQF2zDrvtUyQ9J+m2iPh0tgcoImJM0lixDg7QAR2Z1ak323M0FfSnIuL5YvEB2yNF+4ikg+2UCKAJfUd2Tw3hj0vaGREPTmtaJ2mFpPuK5xdbqRCdWr16dWn7kiVLStvvvPPOyn2vX7++tP26666rvO4FCxaUtk9MTFRe97CazW78RZJ+Lelt228Vy+7WVMj/ZvsmSf+R9Mt2SgTQhL5hj4jXJPX6gX5ps+UAaAuXywJJEHYgCcIOJEHYgSQIO5AEt7gCxxmmbAaSI+xAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgST6ht32Itsbbe+0vcP2qmL5vbY/sv1W8biy/XIBVNV3kgjbI5JGIuJN26dKekPSNZJ+JelwRNw/686YJAJoXa9JImYzP/ukpMni9We2d0pa0Gx5ANr2rX6z2z5b0vmSXi8W3Wp7m+0nbM/t8Z1R2+O2x2tVCqCWWc/1ZvsUSf+U9MeIeN72fEmfSApJf9DUrv6NfdbBbjzQsl678bMKu+05kl6S9HJEPDhD+9mSXoqIH/VZD2EHWlZ5YkfblvS4pJ3Tg14cuDviWknb6xYJoD2zORp/saTNkt6W9FWx+G5JyyWdp6nd+L2SVhYH88rWxcgOtKzWbnxTCDvQPuZnB5Ij7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJNH3D0427BNJ/572/vRi2TAa1tqGtS6J2qpqsrbv9WoY6P3s3+jcHo+IpZ0VUGJYaxvWuiRqq2pQtbEbDyRB2IEkug77WMf9lxnW2oa1LonaqhpIbZ3+ZgcwOF2P7AAGhLADSXQSdttX2H7P9m7bd3VRQy+299p+u5iGutP56Yo59A7a3j5t2Tzbr9jeVTzPOMdeR7UNxTTeJdOMd7rtup7+fOC/2W2fKOl9SZdJmpC0RdLyiHhnoIX0YHuvpKUR0fkFGLZ/JumwpL8cmVrL9p8kHYqI+4r/KOdGxG+HpLZ79S2n8W6ptl7TjP9GHW67Jqc/r6KLkf0CSbsj4oOI+FzSM5KWdVDH0IuITZIOHbV4maQ1xes1mvrHMnA9ahsKETEZEW8Wrz+TdGSa8U63XUldA9FF2BdI2jft/YSGa773kLTe9hu2R7suZgbzj0yzVTyf0XE9R+s7jfcgHTXN+NBsuyrTn9fVRdhnmppmmM7/XRQRP5H0C0m3FLurmJ2HJS3W1ByAk5Ie6LKYYprx5yTdFhGfdlnLdDPUNZDt1kXYJyQtmvZ+oaT9HdQxo4jYXzwflPSCpn52DJMDR2bQLZ4PdlzP/0XEgYj4MiK+kvSoOtx2xTTjz0l6KiKeLxZ3vu1mqmtQ262LsG+RtMT2Oba/I+l6Ses6qOMbbJ9cHDiR7ZMlXa7hm4p6naQVxesVkl7ssJavGZZpvHtNM66Ot13n059HxMAfkq7U1BH5PZJ+10UNPeo6V9LW4rGj69okrdXUbt1/NbVHdJOk70raIGlX8TxviGr7q6am9t6mqWCNdFTbxZr6abhN0lvF48qut11JXQPZblwuCyTBFXRAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kMT/AAGTrcvtVnQkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAM2ElEQVR4nO3dT4wc9ZnG8ecBYg5OQGb5swZ719nAYYOltZExazlaWIVEBg4mh6xiAfJK0U5AASVSDmuxBw8nrNUm0Z4iJgLFWQJR5ITFiEBiWZa8HEAMyDFDTMIfeePJmDGGg8kBezHvHqYcDaaretxV1dX2+/1Ire6ut6vqpYfHVd3VVT9HhACc+87rugEAw0HYgSQIO5AEYQeSIOxAEhcMc2W2+eofaFlEuNf0Wlt22xts/872G7a31FkWgHZ50OPsts+X9HtJX5I0LelFSZsi4rcV87BlB1rWxpZ9raQ3IuKtiDgh6aeSNtZYHoAW1Qn7VZIOzXs+XUz7GNtjtidtT9ZYF4Ca6nxB12tX4RO76RExIWlCYjce6FKdLfu0pOXzni+TNFOvHQBtqRP2FyVdY/uzthdJ+pqknc20BaBpA+/GR8SHtu+V9CtJ50t6JCJebawzAI0a+NDbQCvjMzvQulZ+VAPg7EHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSGLg8dklyfZBSe9LOinpw4hY00RTAJpXK+yFf4yIow0sB0CL2I0Hkqgb9pD0a9sv2R7r9QLbY7YnbU/WXBeAGhwRg89sXxkRM7Yvl7RL0n0Rsbfi9YOvDMCCRIR7Ta+1ZY+ImeL+iKQnJK2tszwA7Rk47LYX2/7MqceSvixpqqnGADSrzrfxV0h6wvap5TwWEc820hU+5plnnqmsb9iwYeBlv/baa5X11atXV9Y/+OCDgdeN4Ro47BHxlqS/a7AXAC3i0BuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0nUulLNGa+MK9X0ND4+Xlnv9zcqTjPu6dprr62cd+XKlZV1TnHt7eqrr66s33nnnZX1fn/zOlq5Ug2AswdhB5Ig7EAShB1IgrADSRB2IAnCDiTBcfYR0O9vsHXr1sr6Y489Vlpbs6Z6YN1+yx7l4+znnVe9rbrxxhtLa4sXL6617uuuu66y/sADD1TWq34bURfH2YHkCDuQBGEHkiDsQBKEHUiCsANJEHYgiTpDNmNI+h2zvfDCC0trhw4darqdM3LxxReX1uoe6160aFFlfWJiorTW73z0fo4fP15Zn5mZqbX8NvTdstt+xPYR21Pzpl1ie5ft14v7Je22CaCuhezG/0jShtOmbZG0OyKukbS7eA5ghPUNe0TslfTeaZM3StpePN4u6faG+wLQsEE/s18REYclKSIO27687IW2xySNDbgeAA1p/Qu6iJiQNCFxIgzQpUEPvc3aXipJxf2R5loC0IZBw75T0ubi8WZJTzbTDoC29D2f3fbjkm6SdKmkWUlbJf23pJ9J+itJf5D01Yg4/Uu8XstiN76HutcUOHbsWGnt5MmTlfPOzs5W1uuez75t27bS2l133VU5bz/9zgm/7LLLSmsXXFDvE+yePXsq6/2uG9/mcfiy89n7/hdHxKaS0hdrdQRgqPi5LJAEYQeSIOxAEoQdSIKwA0lwius54KKLLiqt7d27t3LefkMHnzhxYpCW/qzqFNcrr7yy1rL79XbfffeV1qanp2ut+5133qmsn5WnuAI4NxB2IAnCDiRB2IEkCDuQBGEHkiDsQBIcZz8LPProo5X1N998s7Q2NTVVWpP6n6pZ19NPP11aW7ZsWeW8q1atqqw/9NBDlfUdO3aU1o4ePVo577mILTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJNH3UtKNroxLSffU729wyy23VNafffbZJtsZmrvvvruyvnnz5sr6unXrmmznnFF2KWm27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOezozPD/I0HFrBlt/2I7SO2p+ZNG7f9R9v7itut7bYJoK6F7Mb/SNKGHtO/HxGritsvm20LQNP6hj0i9kp6bwi9AGhRnS/o7rW9v9jNX1L2IttjtidtT9ZYF4CaBg37DyR9TtIqSYclfbfshRExERFrImLNgOsC0ICBwh4RsxFxMiI+kvRDSWubbQtA0wYKu+2l855+RVL19YoBdK7vcXbbj0u6SdKltqclbZV0k+1VkkLSQUnfaLFHAA3oG/aI2NRj8sMt9AKgRfxcFkiCsANJEHYgCcIOJEHYgSQ4xRWdsXte8RgtYcsOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJcN34ETAzM1NZP378+JA6Ga6I6LqFVPpu2W0vt73H9gHbr9r+VjH9Etu7bL9e3C9pv10Ag1rIbvyHkr4TEX8r6e8lfdP25yVtkbQ7Iq6RtLt4DmBE9Q17RByOiJeLx+9LOiDpKkkbJW0vXrZd0u1tNQmgvjP6zG57haTVkl6QdEVEHJbm/kGwfXnJPGOSxuq1CaCuBYfd9qcl/VzStyPi2EIH5YuICUkTxTL4RgboyIIOvdn+lOaC/pOI+EUxedb20qK+VNKRdloE0IS+W3bPbcIflnQgIr43r7RT0mZJ24r7J1vpMIHrr7++sv7uu+8OqROcyxayG79e0l2SXrG9r5h2v+ZC/jPbX5f0B0lfbadFAE3oG/aIeE5S2Qf0LzbbDoC28HNZIAnCDiRB2IEkCDuQBGEHkvAwTzPkF3SDGR8fr6xPTU2V1nbs2NFwN2fmnnvuKa2tWLGict6nnnqqsv7cc88N0tI5LyJ6Hj1jyw4kQdiBJAg7kARhB5Ig7EAShB1IgrADSXAp6bPADTfcUFlfv359aW3lypVNt3NGbrvtttLa5ORk5bwcR28WW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILj7GeBF154obJ+xx13lNZuvvnmptv5mH7Hwt9+++3S2v79+5tuBxXYsgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEn2vG297uaQfS/pLSR9JmoiI/7Q9LulfJL1TvPT+iPhln2Vx3fgWPPjgg6W1LVu2tLrudevWVdaff/75VtePTyq7bvxCflTzoaTvRMTLtj8j6SXbu4ra9yPiP5pqEkB7FjI++2FJh4vH79s+IOmqthsD0Kwz+sxue4Wk1ZJO/X7zXtv7bT9ie0nJPGO2J21XX4MIQKsWHHbbn5b0c0nfjohjkn4g6XOSVmluy//dXvNFxERErImINQ30C2BACwq77U9pLug/iYhfSFJEzEbEyYj4SNIPJa1tr00AdfUNu21LeljSgYj43rzpS+e97CuSyocSBdC5hRx6+4Kk/5H0iuYOvUnS/ZI2aW4XPiQdlPSN4su8qmVx6A1oWdmhN8ZnB84xjM8OJEfYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IYthDNh+V9L/znl9aTBtFo9rbqPYl0dugmuztr8sKQz2f/RMrtydH9dp0o9rbqPYl0dughtUbu/FAEoQdSKLrsE90vP4qo9rbqPYl0dughtJbp5/ZAQxP11t2AENC2IEkOgm77Q22f2f7Ddvtjil8hmwftP2K7X1dj09XjKF3xPbUvGmX2N5l+/XivucYex31Nm77j8V7t8/2rR31ttz2HtsHbL9q+1vF9E7fu4q+hvK+Df0zu+3zJf1e0pckTUt6UdKmiPjtUBspYfugpDUR0fkPMGz/g6Q/SfpxRKwspv27pPciYlvxD+WSiPjXEeltXNKfuh7GuxitaOn8YcYl3S7pn9Xhe1fR1z9pCO9bF1v2tZLeiIi3IuKEpJ9K2thBHyMvIvZKeu+0yRslbS8eb9fc/yxDV9LbSIiIwxHxcvH4fUmnhhnv9L2r6Gsougj7VZIOzXs+rdEa7z0k/dr2S7bHum6mhytODbNV3F/ecT+n6zuM9zCdNsz4yLx3gwx/XlcXYe81NM0oHf9bHxHXSbpF0jeL3VUszIKG8R6WHsOMj4RBhz+vq4uwT0taPu/5MkkzHfTRU0TMFPdHJD2h0RuKevbUCLrF/ZGO+/mzURrGu9cw4xqB967L4c+7CPuLkq6x/VnbiyR9TdLODvr4BNuLiy9OZHuxpC9r9Iai3ilpc/F4s6QnO+zlY0ZlGO+yYcbV8XvX+fDnETH0m6RbNfeN/JuS/q2LHkr6+htJvylur3bdm6THNbdb93+a2yP6uqS/kLRb0uvF/SUj1Nt/aW5o7/2aC9bSjnr7guY+Gu6XtK+43dr1e1fR11DeN34uCyTBL+iAJAg7kARhB5Ig7EAShB1IgrADSRB2IIn/B7Q9EAtCrf0DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMcElEQVR4nO3dX6gc9RnG8eepNiBRMKmYxiRtbPWipdBYglT8g0USojexF63NhaRUe7xoQLEX1RRRKAUpNaUqCCcoSaSNBow1lto0BKkWRHOU6IlNW/8k1Zhj0pALzY02+vbiTMox7s6e7MzsrOf9fmDZ3Xl3Z14mec7M7uzMzxEhADPf59puAMBgEHYgCcIOJEHYgSQIO5DE6YNcmG2++gcaFhHuNL3Slt32Ctv/tP267duqzAtAs9zvcXbbp0n6l6Rlkg5I2iVpVUT8veQ9bNmBhjWxZb9Y0usR8WZEfCjpEUkrK8wPQIOqhH2BpLenPD9QTPsE2yO2x2yPVVgWgIqqfEHXaVfhU7vpETEqaVRiNx5oU5Ut+wFJi6Y8XyjpYLV2ADSlSth3SbrQ9vm2Z0n6gaRt9bQFoG5978ZHxHHbayRtl3SapIci4tXaOgNQq74PvfW1MD6zA41r5Ec1AD47CDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUii7yGbgaZt2rSptH799dc3tmy740Con2mVwm57v6T3JX0k6XhELK2jKQD1q2PL/p2IOFLDfAA0iM/sQBJVwx6S/mL7RdsjnV5ge8T2mO2xissCUEHV3fhLI+Kg7XMl7bD9j4h4ZuoLImJU0qgk2Y6KywPQp0pb9og4WNwflvS4pIvraApA/foOu+3Zts868VjSckl76moMQL2q7MbPk/R4cTzydEm/j4g/19IVPjMuuOCC0nqvY+VV5o1T03fYI+JNSd+ssRcADeLQG5AEYQeSIOxAEoQdSIKwA0lwimty1113XWl9xYoVpfW5c+eW1i+55JKutTvuuKP0vVdffXVpfWys/BfYW7ZsKa1nw5YdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JwxOAuHsOVappx++239/3eXsey58yZU1p/8skn+172vffeW1q//PLLS+vHjh0rrT/11FOn3NNMEBEdr4PNlh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkuA4+wxQ5d9wfHy8tL5+/frS+n333df3stEMjrMDyRF2IAnCDiRB2IEkCDuQBGEHkiDsQBJcN36G27dvX2l93bp1pfUNGzbU2A3a1HPLbvsh24dt75kyba7tHbZfK+7Lr3AAoHXT2Y3fIOnkYUFuk7QzIi6UtLN4DmCI9Qx7RDwj6ehJk1dK2lg83ijp2pr7AlCzfj+zz4uICUmKiAnb53Z7oe0RSSN9LgdATRr/gi4iRiWNSpwIA7Sp30Nvh2zPl6Ti/nB9LQFoQr9h3yZpdfF4taQn6mkHQFN6ns9ue7OkKyWdI+mQpDsl/UHSFklfkvSWpO9FxMlf4nWaF7vxDSj7N1y+fHnpe3fs2FF3O2hZt/PZe35mj4hVXUpXVeoIwEDxc1kgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkuh5KelaF8alpBuxZs2arrWFCxeWvnf27Nml9RdeeKG0/vDDD5fWMXjdLiXNlh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkug5iiuG3/3339+1tn379tL39hrSedeuXaX148ePl9Y3b95cWsfg9Nyy237I9mHbe6ZMu8v2O7Z3F7drmm0TQFXT2Y3fIGlFh+m/iYglxe1P9bYFoG49wx4Rz0g6OoBeADSoyhd0a2y/Uuzmz+n2Itsjtsdsj1VYFoCK+g37A5K+KmmJpAlJ93R7YUSMRsTSiFja57IA1KCvsEfEoYj4KCI+lrRe0sX1tgWgbn2F3fb8KU+/K2lPt9cCGA49j7Pb3izpSknn2D4g6U5JV9peIikk7Zd0U4M9okULFiworV9xxRWldY6zD4+eYY+IVR0mP9hALwAaxM9lgSQIO5AEYQeSIOxAEoQdSIJTXGeAxYsXd62dccYZpe89erT8tIdHH320tH7rrbeW1jE82LIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBIM2TwDfPDBB11rp59e/lOKtWvXltbvuafrRYgk9b6UNAaPIZuB5Ag7kARhB5Ig7EAShB1IgrADSRB2IAnOZ58BZs2a1bV2003lV/neunVraZ3j6DMHW3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSILj7DPcvn37SutHjhwZUCdoW88tu+1Ftp+2vdf2q7ZvLqbPtb3D9mvF/Zzm2wXQr+nsxh+X9NOI+Jqkb0v6ie2vS7pN0s6IuFDSzuI5gCHVM+wRMRERLxWP35e0V9ICSSslbSxetlHStU01CaC6U/rMbnuxpIskPS9pXkRMSJN/EGyf2+U9I5JGqrUJoKpph932mZIek3RLRLxnd7ym3adExKik0WIeXHASaMm0Dr3Z/rwmg/67iDhxmtQh2/OL+nxJh5tpEUAdpvNtvCU9KGlvRKybUtomaXXxeLWkJ+pvD1XZLr0hj+nsxl8q6XpJ47Z3F9PWSrpb0hbbN0h6S9L3mmkRQB16hj0i/iap2ybgqnrbAdAUfi4LJEHYgSQIO5AEYQeSIOxAEpziOsMtW7astH7eeecNqJNT9+yzz5bW33jjjQF1MjOwZQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJDjOPgM899xzXWs33nhj6XvPPvvsSss+duxYaX18fLzveb/77ruldY6znxq27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQhCMGN0gLI8IM3vbt20vry5cvrzT/l19+ubS+ZMmSSvPHqYuIjleDZssOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0n0PM5ue5GkTZK+KOljSaMR8Vvbd0n6saT/FC9dGxF/6jEvjrMDDet2nH06YZ8vaX5EvGT7LEkvSrpW0vclHYuIX0+3CcIONK9b2KczPvuEpIni8fu290paUG97AJp2Sp/ZbS+WdJGk54tJa2y/Yvsh23O6vGfE9pjtsUqdAqhk2r+Nt32mpL9K+mVEbLU9T9IRSSHpF5rc1f9Rj3mwGw80rO/P7JJk+/OS/ihpe0Ss61BfLOmPEfGNHvMh7EDD+j4RxrYlPShp79SgF1/cnfBdSXuqNgmgOdP5Nv4ySc9KGtfkoTdJWitplaQlmtyN3y/ppuLLvLJ5sWUHGlZpN74uhB1oHuezA8kRdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkuh5wcmaHZH07ynPzymmDaNh7W1Y+5LorV919vblboWBns/+qYXbYxGxtLUGSgxrb8Pal0Rv/RpUb+zGA0kQdiCJtsM+2vLyywxrb8Pal0Rv/RpIb61+ZgcwOG1v2QEMCGEHkmgl7LZX2P6n7ddt39ZGD93Y3m973PbutsenK8bQO2x7z5Rpc23vsP1acd9xjL2WervL9jvFuttt+5qWeltk+2nbe22/avvmYnqr666kr4Gst4F/Zrd9mqR/SVom6YCkXZJWRcTfB9pIF7b3S1oaEa3/AMP2FZKOSdp0Ymgt27+SdDQi7i7+UM6JiJ8NSW936RSH8W6ot27DjP9QLa67Ooc/70cbW/aLJb0eEW9GxIeSHpG0soU+hl5EPCPp6EmTV0raWDzeqMn/LAPXpbehEBETEfFS8fh9SSeGGW913ZX0NRBthH2BpLenPD+g4RrvPST9xfaLtkfabqaDeSeG2Sruz225n5P1HMZ7kE4aZnxo1l0/w59X1UbYOw1NM0zH/y6NiG9JulrST4rdVUzPA5K+qskxACck3dNmM8Uw449JuiUi3muzl6k69DWQ9dZG2A9IWjTl+UJJB1voo6OIOFjcH5b0uCY/dgyTQydG0C3uD7fcz/9FxKGI+CgiPpa0Xi2uu2KY8cck/S4ithaTW193nfoa1HprI+y7JF1o+3zbsyT9QNK2Fvr4FNuziy9OZHu2pOUavqGot0laXTxeLemJFnv5hGEZxrvbMONqed21Pvx5RAz8JukaTX4j/4akn7fRQ5e+viLp5eL2atu9Sdqsyd26/2pyj+gGSV+QtFPSa8X93CHq7WFNDu39iiaDNb+l3i7T5EfDVyTtLm7XtL3uSvoayHrj57JAEvyCDkiCsANJEHYgCcIOJEHYgSQIO5AEYQeS+B/8UOkDFR0bJwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Show first 10 images\n",
    "%matplotlib inline \n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "for i in range(10):\n",
    "    image = np.reshape(X[i], (28,28))\n",
    "    plt.imshow(image, cmap=plt.get_cmap('gray'))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(42000, 28, 28, 1)\n",
      "(18000, 28, 28, 1)\n",
      "(42000, 10)\n",
      "(18000, 10)\n"
     ]
    }
   ],
   "source": [
    "# Create 4d tensors\n",
    "Xtrain = Xtrain.reshape(-1, 28, 28, 1) # TensorFlow read -1 as n\n",
    "Xtest = Xtest.reshape(-1, 28, 28, 1)   # TensorFlow read -1 as n\n",
    "print(Xtrain.shape)\n",
    "print(Xtest.shape)\n",
    "\n",
    "ytrain = to_categorical(ytrain)\n",
    "ytest = to_categorical(ytest)\n",
    "print(ytrain.shape)\n",
    "print(ytest.shape)\n",
    "\n",
    "# Create input shape variable for parameters in the model\n",
    "image_rows = 28\n",
    "image_cols = 28\n",
    "colors = 1\n",
    "input_shape = (image_rows, image_cols, colors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating the CNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Dense, Flatten, Dropout, Conv2D, MaxPooling2D\n",
    "from keras.models import Sequential \n",
    "\n",
    "def getModel():\n",
    "    # Instantiate the model\n",
    "    model = Sequential()\n",
    "\n",
    "    # Conv2D takes n dimensional image and makes it 2d\n",
    "    # Filter parameter recommended using powers of 2\n",
    "    # Kernel_size parameter is (1,1) a tuple of odd integers. Recommneded is 1,3,5,7\n",
    "    # Strides parameter is default (1,1) \n",
    "    # Padding parameter is \"valid\" or \"same\". \"valid\" means spatial dimensions are allowed to reduce. You can instead preserve spatial dimensions of the volume such that the output volume size matches the input volume size, by setting the value to “same”. \n",
    "    # Activation parameter is the activation function. Normally use \"relu\" to map negative values to 0\n",
    "    model.add(Conv2D(4, kernel_size=(3, 3),\n",
    "                 activation='relu',\n",
    "                 input_shape=input_shape,\n",
    "                 padding = 'same'))\n",
    "    \n",
    "    # Pooling layer reduces dimensionality of the feature maps but keeps the most important information\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    \n",
    "    # It seems common to increase by a power of 2 filters each Conv2d layers, if max pooling occurs directly after \n",
    "    model.add(Conv2D(8, kernel_size=(3, 3), \n",
    "                     activation='relu',\n",
    "                     padding = 'same'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    \n",
    "    # Flatten layer takes the 2d image from Conv2d and makes it 1d array\n",
    "    model.add(Dropout(0.25))\n",
    "    model.add(Flatten())\n",
    "\n",
    "    # Dense layers\n",
    "    model.add(Dense(128, activation='relu'))\n",
    "    model.add(Dropout(0.5))\n",
    "\n",
    "    # Output layer\n",
    "    model.add(Dense(10, activation = 'softmax'))\n",
    "    \n",
    "    # Compile\n",
    "    model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fit the model\n",
    "from keras.callbacks import EarlyStopping\n",
    "\n",
    "model_original = getModel()\n",
    "\n",
    "# Create early stopping parameter\n",
    "# es = EarlyStopping(monitor='accuracy', mode='max', verbose=1, patience = 10)\n",
    "# Early stopping isn't really necessary since I am only using 10 epochs\n",
    "\n",
    "history_original = model_original.fit(Xtrain, ytrain,\n",
    "                                     epochs = 20,\n",
    "                                     verbose = 0,\n",
    "                                     #callbacks = [es],\n",
    "                                     validation_data  = (Xtest, ytest),\n",
    "                                     batch_size = 32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Augmenting Data with ImageDataGenerator()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU4AAAD7CAYAAAAFI30bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZAV1dnH8e8jEUgJluA6IiIYtKDcJajRGH2RChoTjUYjoVQiimu5lBsSxRi1QhkTl3IpMSIYUbQUlXItpTTEaBQwboAERAXKCaioEDdEzvvHvaf7zjB35vZdepvfp2rq9u3uoQ/3mTnz9OmzmHMOERGp3CZJF0BEJGtUcYqIRKSKU0QkIlWcIiIRqeIUEYlIFaeISEQ1VZxmNsLMFpnZEjMbV69CSbIU1/xSbOvDqu3HaWZdgP8Aw4EVwBxgpHNuQf2KJ3FTXPNLsa2f79XwvUOBJc65pQBmNh04CigbBDPr7L3tP3bObZ10ITqguEaXhbhCxNgqruXjWsuteh9gecn7FcV9Ut4HSRegAoprdFmIKyi2UZWNay0Zp7Wxb6O/UGY2Fhhbw3UkXoprfnUYW8W1MrVUnCuAviXvdwA+bH2Sc24SMAmU+meE4ppfHcZWca1MLbfqc4CBZtbfzLoCJwAz61MsSZDiml+KbZ1UnXE659ab2TnAM0AXYLJzbn7dSiaJUFzzS7Gtn6q7I1V1MaX+85xzQ5IuRL0proprTpWNay1tnCIiDXfxxRcH2xdccAEATU1NLc559NFHg+3f//73ALzxxhsNK5OGXIqIRKSMU0RSZcKECQD89re/BWCbbbYJjt17770A3HbbbQDst99+AAwYMCDOIirjFBGJKhMZ54477gjA5MmTARg2bFhwbPXq1QDceOONAFx99dUxl07y4umnnw62fTvZv/71r4RK07lceeWVwfZZZ50FwLRp0wC45557gmPLli0D4PPPPwdg8ODBAHz33XexlNNTxikiEpEqThGRiFJ3q77TTjsBMHXq1GBf376FUWLbbbfdRudvscUWAJxyyikArF+/HoA//vGPjSympMgmmxT+/vvbNoAePXq0ee6HH4YjDJubmwHo168fAD179tzo35TG8k1rn332WbDv8MMPB2DFihUAfPLJJ8GxuG/Jy9FPh4hIRKnJOH2m+cADDwCw9dbhNHjXXnstAO+++y4QZpkAo0ePBuDII48EYMyYMRv928o+86179+4ATJo0Kdg3cOBAAD744IMW72+//fbgnBtuuAGAQw89FGjZqdqsrYmEpN783eSqVauCfQsWFKYH/frrrxMpUyWUcYqIRJSajNO3SQ0dOhSApUuXBsfmzZsHwOuvvw5At27dgmO+bWTDhg0A/OIXvwDCNk8I20Wuu+66hpRdkjVq1CgAnnzyyWDf8uWF+XoXLVoEhD8PpW1p3p133gnAscceG+xLS1uapJMyThGRiFRxiohE1OGtuplNBo4EVjnndivu6w08AOwEvA8c75z7tHHFbOmbb74Jtv3IDt+Y77uR+C4NAIcccgigW/VSaYxrtfwtth/tAxuP+Bk0aBAQPkjKsyzF9pZbbgHgmGOOCfYNHz4cgOeeew6Ar776quz3/+Mf/wDgzTffDPb5bkyNVEnGOQUY0WrfOGCWc24gMKv4XrJlCoprXk1BsW2oDjNO59xsM9up1e6jgEOK21OBF4BL61GgTz8t/BH03ZKgZVeF1nyXhZdeeqnsOV27dq1H0XIl7rhKfLIU27lz5wJw0kknBfv69+8PwPe+V756OuiggwDYaqutAHj11VeDY34ceyNV+1R9W+dcM4BzrtnMtil3olbNyxTFNb8qiq3iWpmGd0eKumreRx99BITz7UHLYXLl+HbPV155BYDp06cHxy69tPCH1bd7PvXUUxWVXcrLymqIPuaXXHIJAI899liSxUm9pOL60EMPBds+Y2yvA7wf0OAHzrzzzjvBsUrqi1pV+1R9pZk1ARRfy99LS5Yorvml2NZRtRnnTOBkYGLxNTV/xn3HZT9PJ0CXLl2A8Km6Ms6yUhvXqHy79pZbbgnAfffdB7ScPGblypVAON/jLrvsEhzzk8XkSKpjO3v27IrO23///YFw2PW///1vILxTjUuHGaeZ3Q+8DOxqZivMbAyFD3+4mS0GhhffS4Yorvml2DZeJU/VR5Y5NKzMfskAxTW/FNvGS81Y9XrxD4leeOGFYJ9/OKQO8J3HXnvtBYSxnzNnzkbn+FtzPxNX6fwG/hZQ0mXIkMIy535ui7fffhtoOWdnHDTkUkQkotxlnNL5+FmRfvzjHwf7/JKya9euBcKM03djATj//PMBWLduHQAff/xxcMzPtiXJ22effYLtTTfdFAi7H7U121UclHGKiESkjFMy769//SsQTgoBsP322wPh6gF+WVmfZUKYrfz5z38GWnaidi61ffo7jT333BNoGTM/o//DDz8MtLxLiJMyThGRiFRxiohEpFt1yTzfBW3ixLBP9xVXXAGEi/ltvvnmALz22mvBOX6xNt+lJYejhTLt6KOPBmDnnXcO9vm5KL788stEyuQp4xQRiUgZp2Sen5/giSeeCPZ9++23ANx4441A2Nm99Jzm5mZAmWba/OAHPwDCmY9KF+GbMWMGkNxDIU8Zp4hIRMo4JTdKl/T1Kwn49Wr8MT9TVuttSY9TTz0VCIdX+mXBIRysUC++Q72fdR6gd+/ewMbrVpVSxikiElHuMk7/F2SPPfYI9p122mlJFUdi5Du9Axx22GEA/OpXvwJgt912A2DChAnBOb5t8/bbbwdg+fLlwTF1gI9XU1NTsO3XEfJzp957773BMT+ZR69evYBwgpZq1xXz17r88suDfcOGFSaR8ivntqWS+Tj7mtnzZrbQzOab2XnF/b3N7FkzW1x87VVVySURims+Ka7xqORWfT1woXNuELA/cLaZDUbLjWad4ppPimsMKpnIuBnwq+OtNbOFQB8atNyoT8F//etfB/v80gaVTI/vb9VLZ8HxczPeeuuttRYvN+KOaxxKZ8p5/PHHgXBp6UcffXSj88ePHw+EDx9Ku7gk3cG6WlmN62WXXRZsjxo1CoA77rgDgL59+wbHevbsCcDxxx8PwFlnnQVAv379Nvo3/cAI/6AQ2l8Aznv//fc7PCdSG2dxrea9gVfQcqO5objmk+LaOBVXnGbWA3gYON85t6a9htNSlS436pcE9UOq/FA5gFmzZgHtZ5x+8foBAwYAMHTo0OCYn/XGz4IjoUbHNU5+Dk6AsWMLv/t+mWk/rLI08zzooIMA+OKLL+IqYmzyENfzzjsPgNNPPz3Y57uXtZ4vta0Z4F9++WUArr766mDfq6++WpeyVdQdycw2pRCEac65GcXdWm404xTXfFJcG6/DjNMKf6ruAhY65/5Scqiuy436riAXXnghELZrAuy7775AuCRoW77//e8DMGLECACOOeaY4Ni0adNqKVouxRXXtDn88MOD7QMOOACoXxaSBnmM68033xxs+yG0frhsUiq5VT8QOBF4y8x8F/7xFALwYHHp0WXAcY0pojSI4ppPimsMKnmq/iJQroFEy41mlOKaT4prPFI3csiPRS1t/PXdiLp161b2+/z5K1euBFreno8bpy5rnc3gwYOBcNSJ717Up0+f4JyHHnoIgLlz5wKVdVWRxjj33HPb3E4rjVUXEYnI4hyTG6V7g+9WBOHCTL4je1vWrFkDwN133w20XOApReY554YkXYh6S7rbirfJJmEe4DtKH3HEEUCYgfosE2DRokVA2FG6ht8FxTWfysZVGaeISESpzThzSplJPimu+aSMU0SkXlRxiohEpIpTRCQiVZwiIhGp4hQRiUgVp4hIRHEPufwY+KL4mjVbUXu5N56mOh8U13xSXMuItR8ngJnNzWKft6yWOy5Z/XyyWu64ZPXzaXS5dasuIhKRKk4RkYiSqDgnJXDNeshqueOS1c8nq+WOS1Y/n4aWO/Y2ThGRrNOtuohIRKo4RUQiiq3iNLMRZrbIzJaYWWrXsjCzvmb2vJktNLP5ZnZecX9vM3vWzBYXX3slXda0yEJsFdfoFNd2rhtHG6eZdQH+AwwHVgBzgJHOuQUNv3hExTWnm5xzr5lZT2AecDQwGljtnJtY/CHq5Zy7NMGipkJWYqu4RqO4ti+ujHMosMQ5t9Q5tw6YDhwV07Ujcc41O+deK26vBRYCfSiUd2rxtKkUgiMZia3iGpni2o6aKs4IqXwfYHnJ+xXFfalmZjsBewOvANs655qhECxgm+RK1lgRb9EyF9vOGlfI9+9snHGtuuIspvK3AocDg4GRZja43Olt7Et1Pygz6wE8DJzvnFuTdHniEjGukLHYdta4Qr5/Z2OPq3Ouqi/gAOCZkveXAZe1dy6FD74zf31U7ecd11eUuJacn/TnmvRX6uNa5e9s0p9r0l9l41rL7EhtpfL7tT7JzMYCY4Hda7hWXnyQdAEqEDWuko24QgWxVVxbKBvXWto4K0rlnXOTXGGWkl/WcC2JT6S4ugzOnNOJdRhbxbUytVScK4C+Je93AD4sd7Jz7skariXxiRRXyRTFtk5qqTjnAAPNrL+ZdQVOAGbWp1iSIMU1vxTbOqm6jdM5t97MzqHw0KcLMNk5N79uJZNEKK75pdjWT6yzI5lZfBdLp3l5bDtSXBXXnCobV03yISISkSpOEZGIVHGKiESkilNEJKK411UXyZRp06YB8PjjjwNw//33b3Rs1KhR8RdMEqWMU0QkIlWcIiIRZepW/YUXXgDgoIMOCvaddtppQHjbtG7dutjLJSKdizJOEZGIMpFx/v3vfwdgv/0KM2B16dIlOHbzzTcD0L17dwAmT54MwDfffFP23xs6dGiw7Rv7v/76ayDMZj/99NO6lF3qy99ZAPzmN78pe54/VvowR/KnR48eAKxduzbS9/3vf/8DoGfPnlVdVxmniEhEqc04586dG2xffPHFALz33nsbnXfdddcBsMkmhb8BZm1NOdjS974X/rd79+4NhH953njjDQD22muv4JzVq1dHKrvUn88028syS913330tXpWBZke1WWRrbWWV/t9ubm6u6d9WxikiEpEqThGRiDq8VTezycCRwCrn3G7Ffb2BB4CdgPeB451zdX2asuOOOwbbn3zyCQBXXXUVED4AArjkkksAOP300wE46aSTALjnnnuCc/yDn7a8++67AFxzzTUATJkyBWj5ACqPkoprVO3dord3+x311j5P0hZbf3sMld1+t77FLv2eah7qlF7f36JX+1DIqyTjnAKMaLVvHDDLOTcQmFV8L9kyBcU1r6ag2DZUhxmnc252caH3UkcBhxS3pwIvAJfWsVwt/O1vfwOgf//+AMyYMSM49tJLLwHQrVs3ALbccksgfFjUljfffDPYHj16NACfffYZAOvXr69TqdMtDXGtRmkG2d6Dntbjx/1DolLVPCgq7Q7lx6+nTVZj67V+gFPJA9/2+CwVas80vWqfqm/rnGsGcM41m9k25U7UcqOZorjmV0WxVVwr0/DuSM65ScAkqH4q/j322AMIuyW9+uqrwbEoGaLv+P7zn/882Ddx4kSgfn+JOot6xDWKarsTaeaiaBoR17YyvkraOtvqlpSW39Nqn6qvNLMmgOLrqvoVSRKkuOaXYltH1WacM4GTgYnF18fqVqKiU045Jdj27Zd+6KV/yg7gF5vzT9qPPvpoAI477rjgnOnTpwOw9dZbA7DvvvsGxzbddNN6Fz3LGh7XrGqrjTStbZxlpCK2PvuspN2yrYzTbyedeXaYcZrZ/cDLwK5mtsLMxlD48Ieb2WJgePG9ZIjiml+KbeNV8lR9ZJlDw+pcFomR4ppfim3jpXasetTboLfffhsIb9UHDRoUHCsdm97adtttB8DYsYUHiZdddhlQ+zhZyadKu0NJ7drq7F7rGPN60ZBLEZGIUptxVss3OlfaaXaLLbYAYPjw4QBce+21QPvDNCV7fLek0g7snjJHiUoZp4hIRLnLOH33JP/aln79+gXbZ555JgCbbbZZYwsmqdN6zk5Jl7bmzky6G5KnjFNEJCJVnCIiEeXuVt0bMmRIsH3RRRcB4exKfiQShKOQ7rzzTgC++uqruIooFah1rHlnnpczq1qPGCod654WyjhFRCLKfMZ52GGHATBw4EAADjjgAAAOPvjg4JwddtgBgPfffx+AZcuWBcf+8Ic/AOF8nJIPlWSaUWZcaqsbkzRWrUv4NpIyThGRiDKZcR566KHB9hVXXAGEsxwtWbIEaJkhvPzyy0C49K8flglhxjpz5kwA1q1b16hiS4OVxrxRbZr1mkleWmprXaI0tm16yjhFRCLKZMZ57rnnBttdu3YFwpnc33vvPQA2bNgQnLNw4UIAvvvuO6DlCpZ/+tOfAJg1axagjDNvWrdjVttWqZnk45Pmtk2vkvk4+5rZ82a20Mzmm9l5xf29zexZM1tcfO3V+OJKvSiu+aS4xqOSW/X1wIXOuUHA/sDZZjYYLTeadYprPimuMahkIuNmwK+Ot9bMFgJ9SHC50ddffz3Ynj17NgAvvvgiAOPGFX4eShubfZejL774AmjZ9cgv/Pbtt982sMTpk8a41lO5W/TSh0a1Ln3hHxSl6eFQ1uLa1vIYaX4o5EVq4yyu1bw38ApabjQ3FNd8Ulwbp+KK08x6AA8D5zvn1lQ632Ujlhu96qqr2rtei9e2LFiwINgeP348ALvssgsA8+fPB6ItO5xlaYprHGqdwb30IVGaHxhlLa5tLSGcZhV1RzKzTSkEYZpzbkZxt5YbzTjFNZ8U18brMOO0wp+qu4CFzrm/lByKfbnRAQMGAPDRRx8F+2pdG6ipqQmAa665BghngJ8zZ05wju/GlCdpiqvUT57i2l77Z9JZaSW36gcCJwJvmZl/KjOeQgAeLC49ugw4rsz3SzoprvmkuMagkqfqLwLlGki03GhGKa75pLjGI1MjhyZPngzA9ddfH+x75plngLA7UdTF2pYuXQrAmDFjAHj66aeBcAw7wOrVq2sptiSodTekWrsgSXzaui33t+3+Nalbdo1VFxGJKFMZZ3taL7b25ZdfBtvtdU3yVq0qPGTcZ5996lswiU1p96DWmWaUuTclXdrqqlS6gFsSlHGKiESUm4zz6quvBsKuSjfddFNwrDT7lM7BZ59p7qQulSkdPu0zzaS7IynjFBGJyCpp/6vbxTI0NK9B5jnnhnR8WrYoroprtXw2Wdpm6QeltDW4JeYO8GXjqoxTRCQiVZwiIhHl5uGQiGRPe7felQ5iSYIyThGRiOLOOD8Gvii+Zs1W1F7ufvUoSAoprvmkuJYR61N1ADObm8UnkFktd1yy+vlktdxxyern0+hy61ZdRCQiVZwiIhElUXFOSuCa9ZDVcsclq59PVssdl6x+Pg0td+xtnCIiWadbdRGRiFRxiohEFFvFaWYjzGyRmS0xs3FxXTcqM+trZs+b2UIzm29m5xX39zazZ81scfG1V9JlTYssxFZxjU5xbee6cbRxmlkX4D/AcGAFMAcY6Zxb0PCLR1Rcc7rJOfeamfUE5gFHA6OB1c65icUfol7OuUsTLGoqZCW2ims0imv74so4hwJLnHNLnXPrgOnAUTFdOxLnXLNz7rXi9lpgIdCHQnmnFk+bSiE4kpHYKq6RKa7tqKnijJDK9wGWl7xfUdyXama2E7A38AqwrXOuGQrBArZJrmSNFfEWLXOx7axxhXz/zsYZ16orzmIqfytwODAYGGlmg8ud3sa+VPeDMrMewMPA+c65NUmXJy4R4woZi21njSvk+3c29rg656r6Ag4Anil5fxlwWXvnUvjgO/PXR9V+3nF9RYlryflJf65Jf6U+rlX+zib9uSb9VTautcyO1FYqv1/rk8xsLDAW2L2Ga+XFB0kXoAJR4yrZiCtUEFvFtYWyca2ljbOiVN45N8kVZin5ZQ3XkvhEiqvL4Mw5nViHsVVcK1NLxbkC6Fvyfgfgw3InO+eerOFaEp9IcZVMUWzrpJaKcw4w0Mz6m1lX4ARgZn2KJQlSXPNLsa2Tqts4nXPrzewcCg99ugCTnXPz61YySYTiml+Kbf1oXfV4af3tfFJc86lsXLXKpYhkxk9/+lMALrjgAgBGjBiRSDk0O5KISETKOCU1evUKJ7C57bbbADj++OMB+PbbbwHo1q1bcM5ZZ50FwO233x5XESUlfOb59NNPA/Fnnso4RUQiyl3G2b17dwB++cuwv/0ZZ5wBwE9+8pNEyiSVGTNmTLC9fv16AC666CIAPv64sET26NGjg3PifLAp6fDMM88AYYbp2zrjpoxTRCQiVZwiIhGl5lZ9n332AWDq1MLco01NTcGxpUuXAjBhwgQgbBCWfBgypNBV7tBDDw32zZgxA4C77rqrxblDhw4Ntr/77rsYSidp1vohEcTzoEgZp4hIRIlnnPvtV5jV6pFHHgHChwBXXnllcM7+++8PQNeuXWMuncTBd0PydxsAL774YlLFkYRV0sk96YdEyjhFRCJKPOP0HZo/+eQTAMaOLcyhum7duuAc3/4pIvnks0wI2yt9VplGyjhFRCJSxSkiElGHt+pmNhk4EljlnNutuK838ACwE/A+cLxz7tNaCvLll18C4cifSy65JDjWr18/AB577LGy37/ZZpsBMGzYMAB+9KMfBcf8mGYJxRXXSrz11ltAy4d/n37a8MvmVppiWw1/i37DDTcA8Xc1qkQlGecUoHVpxwGznHMDgVnF95ItU1Bc82oKim1DdZhxOudmFxd6L3UUcEhxeyrwAnBpPQrU3NwMwPXXXx/s22STQv2+aNGist/XpUsXALbccssWrwDz52uS69bijmt7/vvf/wJw7rnnBvv8oIfHH3+80ZfPnTTFthY+80zjQ6Jqn6pv65xrBnDONZvZNuVO1HKjmaK45ldFsVVcK9Pw7kjOuUnAJGh/Kv6BAwcCcM455wBw3XXXBceWL1/e5vdIciqNaxQ333xzh+csXrw42PZ3J1I/jYhrJUqzyigZZlvnxjFHZ7VP1VeaWRNA8XVV/YokCVJc80uxraNqM86ZwMnAxOJr+cfdHfDtlrfccgsAo0aNAqBv33D553feeQeA6dOnA/D6668Hx3r06AHA8OHDATjppJMAZSNVqltc680PzX3uueeCff7nQiqSyti2Hl4J6Xly3p4OM04zux94GdjVzFaY2RgKH/5wM1sMDC++lwxRXPNLsW28Sp6qjyxzaFidyyIxUlzzS7FtvMTHqq9cuRKAW2+9FQgX5SrttH7IIYcAsP3227c4F2DJkiUA9O/fv8W/WzrTjmTf7rvvDoQ/LxA+KNqwYUMiZZLOS0MuRUQiSjzj9HwmMWnSJCDsFA2w6667AnDggQcCcPnllwfHfMa58847A7BgwQIgnZ1mJTo/e5af7b101iwt1iZJUcYpIhJRajJOz2eed955Z7DPD5884YQTgHDOToAjjzwSgDVr1gBw7733xlJOiceee+4JhJnmu+++GxzzSwiLxE0Zp4hIRKo4RUQiSt2telv8shr3338/EHZZAjjzzDMBGDBgQPwFE5FOSRmniEhEmcg4vW+++QaADz/8MNi3bNkyIOwE7bPTgw8+ODhn9uzZcRVR6sR3Q9p3332BsNuZfxWB5Ma6K+MUEYkoUxnnVlttBcARRxwR7OvduzcAd9xxBxC2f55xxhnBOco4s2fvvfcGwk7u7c125WfS2m233YJ9fjimMlRpBGWcIiIRVbLKZV/gHmA7YAMwyTl3U5Kr5pWugDhjxgwgnKvz2GOPjaMImZfGuP7sZz8Ltn/3u98BcPfddwPw9ttvb3T+qaeeCsCYMWOAlhmnXyW1s2WcaYxrJXxbJcQzg3utKsk41wMXOucGAfsDZ5vZYLRqXtYprvmkuMagw4rTOdfsnHutuL0WWAj0obBqnp+7bSpwdKMKKfWnuOaT4hqPSA+HikuO7g28QoQVEevFd0fyXZAAPv/880ZfNveSjqs3aNCgYPumm24C4IMPPgDgySefBOD//u//gnP8z8GECRMAeOKJJ4JjX3/9dWMLmwFpiWt7/CxmpbflpV2LymndDSnu2/qKK04z6wE8DJzvnFtjZpV+n5YbTTHFNZ8U18aqqOI0s00pBGGac25GcfdKM2sq/vUqu2pePZcb9XN0+q5HpTbffPNa/ulOKem4du3aFQizh9Is0S8T/cMf/hAIZ0U68cQTg3MeeeQRQLMktZZ0XGvlfx4qmW81qXl3K1mszYC7gIXOub+UHPKr5kGKVs2Tyiiu+aS4xqOSjPNA4ETgLTPz6/KOp7BK3oPFFfSWAcc1poihLl26ANCrV69gn18e2L/6uTulQ4nHdeTIwppiPossnajln//8JxCuCOAHMfg2Tykr8bhWozRz9O2VvltSe+cn1WWpklUuXwTKNZBo1byMUlzzSXGNh0YOiYhElKmx6n554HHjwr67p5xyCgDdu3cHwu5JDz74YMylk6j84ns9e/YEYPz48cGxp556ClB3s87I34ZX2hMgCco4RUQisjiXWK1X94ampqZg+6KLLgLCcct+sbazzz67Hpeqt3nOuSFJF6Lekuq2kiKKaz6VjasyThGRiDKZcWaYMpN8UlzzSRmniEi9qOIUEYlIFaeISESqOEVEIlLFKSISkSpOEZGI4h5y+THwRfE1a7ai9nL3q0dBUkhxzSfFtYxY+3ECmNncLPZ5y2q545LVzyer5Y5LVj+fRpdbt+oiIhGp4hQRiSiJinNSAtesh6yWOy5Z/XyyWu64ZPXzaWi5Y2/jFBHJOt2qi4hEFFvFaWYjzGyRmS0xs3Edf0cyzKyvmT1vZgvNbL6ZnVfc39vMnjWzxcXXXh39W51FFmKruEanuLZz3Thu1c2sC/AfYDiwApgDjHTOLWj4xSMqrjnd5Jx7zcx6AvOAo4HRwGrn3MTiD1Ev59ylCRY1FbISW8U1GsW1fXFlnEOBJc65pc65dcB04KiYrh2Jc67ZOfdacXstsBDoQ6G8U4unTaUQHMlIbBXXyBTXdsRVcfYBlpe8X1Hcl2pmthOwN/AKsK1zrhkKwQK2Sa5kqZK52CquFVFc2xFXxdnWcnWpfpxvZj2Ah4HznXNrki5PilTwOVcAAADgSURBVGUqtoprxRTXdsRVca4A+pa83wH4MKZrR2Zmm1IIwjTn3Izi7pXF9hTfrrIqqfKlTGZiq7hGori2I66Kcw4w0Mz6m1lX4ARgZkzXjsQKiznfBSx0zv2l5NBM4OTi9snAY3GXLaUyEVvFNTLFtb3rxtUB3syOAG4EugCTnXPXxnLhiMzsIOAfwFvAhuLu8RTaTR4EdgSWAcc551YnUsiUyUJsFdfoFNd2rquRQyIi0WjkkIhIRKo4RUQiUsUpIhKRKk4RkYhUcYqIRKSKU0QkIlWcIiIRqeIUEYno/wEwX3RzF90aHgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Observe how parameters affect images\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "datagen = ImageDataGenerator(width_shift_range=.2)\n",
    "# fit parameters from data\n",
    "datagen.fit(Xtrain)\n",
    "# configure batch size and retrieve one batch of images\n",
    "for X_batch, y_batch in datagen.flow(Xtrain, ytrain, batch_size=9):\n",
    "    # create a grid of 3x3 images\n",
    "    for i in range(0, 9):\n",
    "        plt.subplot(330 + 1 + i)\n",
    "        plt.imshow(X_batch[i].reshape(28, 28), cmap=plt.get_cmap('gray'))\n",
    "    # show the plot\n",
    "    plt.show()\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU4AAAD7CAYAAAAFI30bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAeeUlEQVR4nO3de7RUZf3H8fdX1FRAxRBF5BdoaF6oCBfiAtfCy0lUEkxlieVS1MhrWpYXWhrJSrxkFkrqSREyw1RUSM1ClqXgJQFTRORSEqFHAW/ghRR7fn/M7Jk5nDPn7Gdmz5699/m81jpr9uzZM/thPod9vvv2POacQ0REwtuq3g0QEUkbbThFRDxpwyki4kkbThERT9pwioh40oZTRMRTVRtOMxtuZsvMbKWZXRZVo6S+lGt2KdtoWKXXcZpZJ2A50ACsAZ4HxjjnXomueRI35ZpdyjY6W1fx3kHASufcvwDM7B5gJFA2BDPr6Ffbr3fO7VrvRrRDufpLQ67gma1yLZ9rNbvqvYD/lDxfk58n5f273g0IQbn6S0OuoGx9lc21morTWpnX4i+UmY0DxlWxHomXcs2udrNVruFUs+FcA/Queb4n8MaWCznnGoFGUOmfEso1u9rNVrmGU82u+vNAPzPra2bbAicDs6NpltSRcs0uZRuRiitO59xmMzsf+DPQCZjqnFsSWcukLpRrdinb6FR8OVJFK1Ppv9A5d1C9GxE15apcM6psrtUc4xQRqYsuXboA8PjjjwNw7LHHFl57++23a75+3XIpIuJJFaeIpMYee+wBwCWXXALA//73PwAGDBhQWCaoQmtJFaeIiKcOVXFed911APzoRz9q8drDDz8MwDe+8Y1Y2yTSER10UO6cy4EHHtjitWnTppV937e//W0AXnzxRQAuuugiAG666abCMqo4RUQSSBtOERFPHWJXPSjjzzzzzLLLfP3rXwdgzpw5ADQ0NNS+YRJacDLg2muvrej9U6dOBdr+HZDaGzhwIFD8P7lixQoAhg8fXlimrV31ci644ILqG+dBFaeIiKfMVpxBhQFw0kknAfDLX/4SgMbGxhbLH3bYYQCcc845MbROfFV7h9uIESMAuPXWWwvzzj777Ko+U/x9+ctfBoqV5mWX5TqhHzRoUNn3BCeSoHhy6MYbbwRg4sSJQPHEL8DGjRsjbHHrVHGKiHjKXMU5Y8YMoPktWMHtWe+++y4Aq1atavG+N998EyheHvHYY48VXis9/iLxGjt2LACjR48G4IUXXii8Nnny5NCfc+ihhwJwwAEHRNg6CeOrX/1qYTq4UP3iiy8GYN26dQAcd9xxhWVeeukloFidBhe9AyxYsACAhx56CIDZs3OdOwV7k3FRxSki4kkbThERT+3uqpvZVGAEsNY5d2B+3i7AH4A+wCpgtHPu3do1M7y9994bgK5du3q979lnnwXgvPPOA+DKK6+MtmEJk5Zchw4dCsDmzZuB5rvnd911V+jPWbx4MQCjRo0qzLv88ssBmDRpUtXtTJKkZdu5c+cW08EuemD58uWF6f79+5f9rOB9wWG3eglTcU4DtjzIdxkw1znXD5ibfy7pMg3lmlXTULY11W7F6Zx70sz6bDF7JDAsPz0d+CtwaYTt8vbb3/4WgH322QeAK664ovDa3//+d6D5X7UtBX/Bgsok65Ke67BhuWb06dMHgCeffBLwqzJLLVy4EIDtttuuMG/evHlA9irOpGebBZWeVd/NOdcE4JxrMrMe5RbUqHmpolyzK1S2yjWcml+OFNeoecHlJjvttBNQrDAA/vKXv7T7/q997WtA8Za+Hj2Kv1f3338/ACeeeGI0jc2AWucaXNgcHOMMLkOR2qpFru+9916L6WBPorVLAyth1trIx7VT6Vn1t8ysJ0D+cW10TZI6Uq7ZpWwjVGnFORs4Dbgm/zgrshZV6aqrrgL8j1WuXZv7PQqOex1++OGF10444YSIWpd4ics1OJv+2Wef1bklqVe3bJcsKQ6k+fLLLwPF26Cvv/76uJoRqXYrTjObATwD7Gtma8zsTHJffoOZrQAa8s8lRZRrdinb2gtzVn1MmZeOiLgtEiPlml3KtvYyd6/6H//4RwDWrFnj9b5g+V//+tdAcRAogAkTJkTTOPH24x//GIDf/OY3dW6JROG5554Dml8W5uPII48Eiv15Bveqf/DBBxG0Ljzdciki4in1FefVV18NQLdu3SL5vLfeeguA6dOnF+ap4qyfpNxiJ9F45ZVXvN8T3A4NsGjRIgDOP/98oHi52qZNmyJoXXiqOEVEPKW+4gw6agiOTQb9aopI8gT9qgY3mowbl7tJKegJHqBv377N3hNcKght3zYdJ1WcIiKetOEUEfGU+l31QFDOf/rpp5F83oYNGwrTN998cySfKW0LDvhD8+ESJN2GDBlSmP79738PFO8EC4ao2WqrYg0X1f3rtaSKU0TEU2YqzuCSoUceeQQoXlbkq3v37kCxP0iIf7D7jiq4qFmyoVOnTgDsvvvuhXk/+9nPAPjJT34CFAddGzlyZNnP2XHHHQvTPXv2jLydlVDFKSLiKTMVZ1SCfgJLe20J+uOU+DQ1NQHw4Ycf1rklUqkddtgBaD48cOnIDGGV7v0Fw37X+zioKk4REU+prziD3t2DHuAlG0455RSgONaQpM/GjRuB1qvMww47DCiOEdaa4OaW/fbbrzAvOCYanIuolzD9cfY2syfMbKmZLTGzC/PzdzGzOWa2Iv8Yzc3iEgvlmk3KNR5hdtU3Axc75/YDBgPnmdn+aLjRtFOu2aRcYxCmI+MmIBgdb6OZLQV6kZDhRo866igAXnvtNaA4mP0///nPwjJhetYJelfaa6+9gGIvLFmV9FyDGxlK+0WV9iUp1+DystLL+YLejCZPngzAnDlzANh5550Ly9x5550AnH766QC88MILhdeC951zzjk1anU4Xsc482M1DwCeQ8ONZoZyzSblWjuhN5xm1gWYCVzknNsQdjjOuIYHXrZsGQBTpkwBmvcw/dRTTwHFnpPWr19feC2oNE8++WQAxo4dC8CgQYNq1dRESXquUpkk5DprVm48uKeffrowLxh2+09/+hMAP/3pT4HiniMUK81DDjkEaD688MCBAwEYMWIEAFOnTq2miRULdTmSmW1DLoS7nXMP5GdruNGUU67ZpFxrr92K03J/qu4AljrnflHyUqKGkh0+fDhQ/Ct3ww03FF7bfvvtAZg0aRIAjY2NhdeCoX8nTpwIFI+5ZF1achU/Sco1ODYZVI4Av/vd7wC45JJLAPj4448BGDx4cGGZ4FbNt99+u8VnHnfccUDx1s1gDKO4hdlVHwKcCiw2s3/k540nF8C9+aFHVwMn1aaJUiPKNZuUawzCnFWfB5Q7QKLhRlNKuWaTco2HORffcf04TyLMmDGjMN3Q0AAU750t9dFHHwHFXfQxY8oNSR2Jhc65g2q5gnqIKtfS38XgspX58+dH8dEFpX1Dzps3D4CwJ07aoFyzqWyuulddRMRT6u9VL6e1yvG6665rMS84SC0iEpYqThERT5mtOFuj6lJEoqCKU0TEU4eqOCXZIji73a7Ss/RxrE+ySRWniIgnbThFRDxpwyki4kkbThERT3GfHFoPfJh/TJvuVN/uL0TRkARSrtmkXMuI9V51ADNbkMb7etPa7rik9ftJa7vjktbvp9bt1q66iIgnbThFRDzVY8PZ2P4iiZTWdsclrd9PWtsdl7R+PzVtd+zHOEVE0k676iIinrThFBHxFNuG08yGm9kyM1tpZpfFtV5fZtbbzJ4ws6VmtsTMLszP38XM5pjZivxjt3q3NSnSkK1y9adc21hvHMc4zawTsBxoANYAzwNjnHOv1HzlnvJjTvd0zi0ys67AQmAUcDrwjnPumvwvUTfn3KV1bGoipCVb5epHubYtropzELDSOfcv59wnwD3AyJjW7cU51+ScW5Sf3ggsBXqRa+/0/GLTyYUjKclWuXpTrm2oasPpUcr3Av5T8nxNfl6imVkfYADwHLCbc64JcmEBPerXstry3EVLXbYdNVfI9v/ZOHOteMOZL+WnAEcD+wNjzGz/cou3Mi/R10GZWRdgJnCRc25DvdsTF89cIWXZdtRcIdv/Z2PP1TlX0Q9wCPDnkueXA5e3tSy5L74j/6yr9PuO68cn15Ll6/291vsn8blW+H+23t9rvX/K5lpN70itlfIHb7mQmY0DxgH9q1hXVvy73g0IwTdXSUeuECJb5dpM2VyrOcYZqpR3zjW6XC8lx1exLomPV64uhT3ndGDtZqtcw6lmw7kG6F3yfE/gjXILO+cerWJdEh+vXCVVlG1EqtlwPg/0M7O+ZrYtcDIwO5pmSR0p1+xSthGp+Binc26zmZ1P7qRPJ2Cqc25JZC2TulCu2aVsoxNr70hmFt/KkmlhFo8dKVflmlFlc1UnH5JaJZfOiMRKG04REU9xj3IpUpXWKsxgnllrV9uIRE8Vp4iIJ1WckgphjmWWLqPqMzkWL15cmD777LMBmD9/flWfOWTIEADmzZsHwBtvFC9H7dWr9n2RqOIUEfGkDaeIiKcOsau+7777AvDqq68C8KUvfanw2rJly+rSJqnelrvjujQpGdavXw/AgQceCMChhx5aeO2DDz6oS5uipopTRMRTZivOoUOHFqYnT57c7LUZM2YUpjdt2gTAlClTALj77rtjaJ1Idn3+858HinsE7733XuG1++67D4Bbb70VgLlz53p99hFHHAHAXXfdBcCKFSsAGDNmTGGZ1157DYC+fft6tz0sVZwiIp4yV3EeffTRAPzwhz8szBswYECzZe69997C9OjRo4F4LmEQ6eh23313ALbffvuK3h+878033wSK/38//PDDwjJ9+vSpooXhqOIUEfGkDaeIiKd2d9XNbCowAljrnDswP28X4A9AH2AVMNo5927tmhle//65oY0OP/zwwry3334bgMbGRgDuuOOOwmuvv/46AHvttRcAxx57LACPPPJI7RtbR2nLVcJLQ7bXX389AJs3bwbgscce83p/sGu+cuVKAHr27Blh69oXpuKcBgzfYt5lwFznXD9gbv65pMs0lGtWTUPZ1lS7Fadz7sn8QO+lRgLD8tPTgb8Cl0bYLm8DBw4Eihfdlt67OnPmTADGjx/f4n3BZQ3XXnstAN/85jeB7Fecack1EFzaUnqRe1sXvHfkHpOSkm3w3Qd7egBTp04FYNy43ECau+66a6TrikulZ9V3c841ATjnmsysR7kFNdxoqijX7AqVrXINp+aXIznnGoFGqG1X/Oeeey4Ap556KgCzZs0qvPa9732vVqvtsOLKdUullYVusYxelLnedtttADzwwAOFebNn58aGO/746kYL32+//QC48cYbAbjuuuuq+jxflZ5Vf8vMegLkH9dG1ySpI+WaXco2QpVWnLOB04Br8o+z2l48Pp999hlQPFvna6utcn9Lttlmm8K8Tz/9tPqGpUNic22NOvnwElu2Y8eObfb8oYceKkyX3n5ZjeC2ziOPPBJIYMVpZjOAZ4B9zWyNmZ1J7stvMLMVQEP+uaSIcs0uZVt7Yc6qjynz0hERt0VipFyzS9nWXubuVb/99tsBuPLKKyt6/4knnghA165dW8wTkfZNmzYttnUFh9Y2bNhQmHfBBRfUfr01X4OISMZkruJ8//33AVi71u+kYXAb5tZb576SQYMGRdswEYlcjx65y1FLq8xrrqn94VtVnCIinlJfcX7/+98HYPDgwQA8/PDDFX3O8uXLAXj55ZcBVZxJo0uNpDXdu3cHVHGKiCSeNpwiIp5Sv6se9GbU1NQEwNNPP13P5kjEKt1F74i9IqXBCSecABQHUmtoaADgpZdeKizz4osvln3/qlWrgGKPZ8HnxU0Vp4iIp9RXnIFgWN/SXpEkO8JUkDqBlEyjRo0qTE+aNAmAfv36AcX+czt16lRYJugd/h//+EeLzwpO3ga9IqniFBFJicxUnCKlOnIP8EkzceLEwnRQaQbnIvbcc08ATjnllMIywU0oV199NdD8mGcwttCwYcOA4q2Wjz76aC2aXpYqThERT6o4JRUqrSBVedbfM888U5ju1asXUDzWGfTkHvSrCcXxwyZMmADALbfcUnjt4IMPBuCqq64Cisc8v/Od79Si6WWF6Y+zt5k9YWZLzWyJmV2Yn7+Lmc0xsxX5x261b65ERblmk3KNR5hd9c3Axc65/YDBwHlmtj8abjTtlGs2KdcYhOnIuAkIRsfbaGZLgV4keChZaV9acm1tWOAthbkMqa1lsrQbn8Rcg6GAobhrHgguPQoeAc4888xmj8Eue6lgF71evI5x5sdqHgA8h4YbzQzlmk3KtXZCbzjNrAswE7jIObch7F/pWgwje//99xemt/wLJn6SlGvI9ZZ9rbW2d5RKc0tJyvWLX/xiYbpz587tLh/0jRs8tmbIkCEA3HrrrVW2rjKhLkcys23IhXC3cy4YJFnDjaaccs0m5Vp77VaclvtTdQew1Dn3i5KX6jaUbL1us8qSJObaljDHOtt6X0eRxFwffPDBwnQwrO/HH38c1+prIsyu+hDgVGCxmQU3j44nF8C9+aFHVwMn1aaJUiPKNZuUawzCnFWfB5T7s63hRlNKuWaTco1HKu8cam340RUrVsTfEIldR9v1zpqTTsoVuvPnz69zS6qje9VFRDylsuIcO3ZszT573bp1AKxevbowLxgI7tlnn63ZekUkPVRxioh4sjh7zY7zQulqBRfYAsybNw+I5PjaQufcQdV+SNKkKdcaUa7ZVDZXVZwiIp5SeYwzDqVn/XQmV0RKqeIUEfGkDaeIiCdtOEVEPGnDKSLiKe6TQ+uBD/OPadOd6tv9hSgakkDKNZuUaxmxXscJYGYL0njNW1rbHZe0fj9pbXdc0vr91Lrd2lUXEfGkDaeIiKd6bDgb67DOKKS13XFJ6/eT1nbHJa3fT03bHfsxThGRtNOuuoiIJ204RUQ8xbbhNLPhZrbMzFaa2WVxrdeXmfU2syfMbKmZLTGzC/PzdzGzOWa2Iv/Yrd5tTYo0ZKtc/SnXNtYbxzFOM+sELAcagDXA88AY59wrNV+5p/yY0z2dc4vMrCuwEBgFnA6845y7Jv9L1M05d2kdm5oIaclWufpRrm2Lq+IcBKx0zv3LOfcJcA8wMqZ1e3HONTnnFuWnNwJLgV7k2js9v9h0cuFISrJVrt6Uaxuq2nB6lPK9gP+UPF+Tn5doZtYHGAA8B+zmnGuCXFhAj/q1rLY8d9FSl21HzRWy/X82zlwr3nDmS/kpwNHA/sAYM9u/3OKtzEv0dVBm1gWYCVzknNtQ7/bExTNXSFm2HTVXyPb/2dhzdc5V9AMcAvy55PnlwOVtLUvui+/IP+sq/b7j+vHJtWT5en+v9f5JfK4V/p+t9/da75+yuVbTO1JrpfzBWy5kZuOAcUD/KtaVFf+udwNC8M1V0pErhMhWuTZTNtdqNpyhSnnnXCPQaGbHAI9UsT6Jh1euoNEQU6TdbOPO9ayzzgKga9euhXk33nhjrVdbtWpODq0Bepc83xN4o9zCzrlHq1iXxMcrV0kVZRuRajaczwP9zKyvmW0LnAzMjqZZUkfKNbuUbUQq3lV3zm02s/PJnfTpBEx1zi2JrGVSF8o1u5KUbefOnQGYPTu33V67dm09mlGxqobOyO9+axc8Y5RrdinbaMQ95pCICMOGDQPg2GOPBeDcc8+tY2v8qXckERFPHbLidK10bGLW2pUaIiItqeIUEfHUISrOLSvM0uqytepTRKQtqjhFRDxpwyki4imzu+phTwAF84LldZIofbbaKvf3f+utW/4633LLLQCcccYZLV476KCDAFi8eHGL1z755JMomygZo4pTRMRTZivOUqois2nbbbcFihdP+/aqs2DBgrKv6XdG2qKKU0TEU4eoOMNQhZFsQXW5xx57FOYdddRRQLHSfP/991u8b6eddgKKnUh89NFHNW2n+Bk5Mjf+21NPPVWYN2PGjHo1JzRVnCIinrThFBHx1O6uuplNBUYAa51zB+bn7QL8AegDrAJGO+ferV0z49NRLktKW6777LMP0PzSoQ0bNjSbd+eddxZe22233QD41re+BcDYsWMBePzxx2vf2DpLU7bBoZfJkycX5k2YMAGAU045BYCFCxfG3q72hKk4pwHDt5h3GTDXOdcPmJt/LukyDeWaVdNQtjXVbsXpnHsyP9B7qZHAsPz0dOCvwKURtqtiHaVirFbacg0ucg+qTIApU6YAMH78+BbLn3POOQCcd955QMeoNANJzvaAAw4A4L777ms2/+abby5M779/bqj37bffPr6Gear0rPpuzrkmAOdck5n1KLeghhtNFeWaXaGyVa7h1PxyJA0jm01x5fq5z30OgP79+wPQ2NhYeK21SjMQ3GopfmqRa+neX5cuXQCYO3cuALNmzQKKew9QvPTspptuimL1NVHpWfW3zKwnQP4xXSMtSTnKNbuUbYQqrThnA6cB1+QfZ0XWIqmnxOXavXt3AC655BIAvvKVr9SzOWlWt2x33HHHwvTf/vY3AKZPnw7A7bffDhRvVAAYNy53pODjjz+Oq4ne2q04zWwG8Aywr5mtMbMzyX35DWa2AmjIP5cUUa7ZpWxrL8xZ9TFlXjoi4rZIjJRrdinb2tO96iJSU0OHDi1ML1++HIDvfve7zZa54YYbWp1OKt1yKSLiSRWnpEJwkuj4448vzHvwwQfr1RzxUJqTb5+pSaWKU0TEkypOSbSNGzcC8OijjwLw85//vMUyqjyTrbQP1EsvTcQdvFVTxSki4kkbThERT9pV34J6VUqWoDekX/3qVwCcddZZhdcmTZrUbFntsktYEydOBOCKK66o6P2qOEVEPKnizFM/nsm08847A633hLTvvvsCcMwxxwCqOKW5oNf/hoaGFq+deOKJgCpOEZHYqOKURAr6ZLztttsAGD16NACvvvpqYZlrr70WKN7GJ8nUuXPnwvTUqVMBOOOMM6r6zGBMqXvuuafsMsE4VaVDSgdGjBhR1fpVcYqIeAozymVv4LfA7sD/gEbn3K+SOmqehJP0XDt16gTAEUfkOvQJjkmtW7eusMyTTz4Zd7MSL4m5Dh48uDA9f/58AIYMGVLVZ269dW7Ttddee7V4LRglYMuOREqtXLmyqvWHqTg3Axc75/YDBgPnmdn+aNS8tFOu2aRcY9DuhtM51+ScW5Sf3ggsBXqRGzVven6x6cCoWjVSoqdcs0m5xsPr5FB+yNEBwHN4jIgoyZbEXGfPng0Uh0+YOXNmPZqRaknJddGiRYXpXXfdtdVlJkyYUJj+wQ9+UPazSofhKOeTTz4B4L///W/IFvoLveE0sy7ATOAi59yGsNc7arjRZFOu2aRca8w51+4PsA3wZ+AHJfOWAT3z0z2BZSE+x8X1UyrMcjG1a0GY7zuunyTnumnTJrdp0yY3cOBAN3DgwNh+b5Rrff6/JvSnbK5hBmsz4A5gqXPuFyUvBaPmQUJGRJTwlGs2Kdd4WP4vS/kFzIYCTwGLyV3eADCe3HGTe4H/A1YDJznn3mnns9peWYTa+3dtKaZbLRc65w6KY0XtSXquQR+OO+ywQ9QfXQvKNZvK5hpmlMt5QLmtikbNSynlmk3KNR66c0hExFNm71Uv3fVua7ddvSEl03bbbQfAM888A8AhhxxSz+aINKOKU0TEU2YrzlKqKtOnd+/eAKxevRqA9evXt/ueyZMnF6avuuqq2jRMBFWcIiLeOkTFKenz+uuvA7D33nsD8Nprr7X7npRcuiQZoIpTRMSTKk5JtFWrVgE6Ti3JoopTRMSTNpwiIp604RQR8aQNp4iIp7hPDq0HPsw/pk13qm/3F6JoSAIp12xSrmW0261c1MxsQVK64PKR1nbHJa3fT1rbHZe0fj+1brd21UVEPGnDKSLiqR4bzsY6rDMKaW13XNL6/aS13XFJ6/dT03bHfoxTRCTttKsuIuIptg2nmQ03s2VmttLMLotrvb7MrLeZPWFmS81siZldmJ+/i5nNMbMV+cdu9W5rUqQhW+XqT7m2sd44dtXNrBOwHGgA1gDPA2Occ6/UfOWezKwnufGnF5lZV2AhMAo4HXjHOXdN/peom3Pu0jo2NRHSkq1y9aNc2xZXxTkIWOmc+5dz7hPgHmBkTOv24pxrcs4tyk9vBJYCvci1d3p+senkwpGUZKtcvSnXNsS14ewF/Kfk+Zr8vEQzsz7AAHJjUu/mnGuCXFhAj/q1LFFSl61yDUW5tiGuDWdrnSkm+nS+mXUBZgIXOec21Ls9CZaqbJVraMq1DXFtONcAvUue7wm8EdO6vZnZNuRCuNs590B+9lv54ynBcZW19WpfwqQmW+XqRbm2Ia4N5/NAPzPra2bbAicDs2NatxfLdTV+B7DUOfeLkpdmA6flp08DZsXdtoRKRbbK1ZtybWu9cV0Ab2bHAL8EOgFTnXM/i2XFnsxsKPAUsBj4X372eHLHTe4F/g9YDZzknHunLo1MmDRkq1z9Kdc21qs7h0RE/OjOIRERT9pwioh40oZTRMSTNpwiIp604RQR8aQNp4iIJ204RUQ8acMpIuLp/wFm8ZwAEn8F4AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "datagen = ImageDataGenerator(samplewise_center=True)\n",
    "# fit parameters from data\n",
    "datagen.fit(Xtrain)\n",
    "# configure batch size and retrieve one batch of images\n",
    "for X_batch, y_batch in datagen.flow(Xtrain, ytrain, batch_size=9):\n",
    "    # create a grid of 3x3 images\n",
    "    for i in range(0, 9):\n",
    "        plt.subplot(330 + 1 + i)\n",
    "        plt.imshow(X_batch[i].reshape(28, 28), cmap=plt.get_cmap('gray'))\n",
    "    # show the plot\n",
    "    plt.show()\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\thomas.m.sugg\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\keras_preprocessing\\image\\image_data_generator.py:348: UserWarning: This ImageDataGenerator specifies `featurewise_std_normalization`, which overrides setting of `featurewise_center`.\n",
      "  warnings.warn('This ImageDataGenerator specifies '\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU4AAAD7CAYAAAAFI30bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZBU1fnG8e8rKqJgBEFEQCUlkihqVFQ0akkiipYp4i6iBkvjFvcloLHikl/UaNRYmqC4gXFfcI0bUQjBICIoUUSEoMEJBIKoIBoJcn5/dJ/uHmZ6pm/37bvN86ma6tu3b/c99MPcee92jjnnEBGRyq0XdwNERNJGG04RkYC04RQRCUgbThGRgLThFBEJSBtOEZGAatpwmtkQM5trZvPNbFRYjZJ4KdfsUrbhsGqv4zSzdsAHwGCgAZgODHPOvRde8yRqyjW7lG141q/hvXsC851zCwDM7GFgKFA2BDNr61fbL3POdYu7Ea1QrsGlIVcImK1yLZ9rLbvqPYGPS5435OdJef+MuwEVUK7BpSFXULZBlc21lorTmpnX5C+UmZ0GnFbDeiRayjW7Ws1WuVamlg1nA9C75HkvYNG6CznnxgBjQKV/SijX7Go1W+VamVp21acDfc2sj5ltCBwHPBNOsyRGyjW7lG1Iqq44nXNrzOxs4CWgHXCPc252aC2TWCjX7FK24an6cqSqVqbSf4ZzbkDcjQibclWuGVU2V905JCISkDacIiIBacMpIhKQNpwiIgHVch1nphxwwAEA3HLLLQDssssuMbZGatWpU6eKl73tttsAOOmkk5q89sUXXwT+PMk+VZwiIgFpwykiElCqdtXnzp1b9rUTTzwRgDfeeCPQZx500EEA3HXXXQBsscUWAEyePLmwzP777x/oMyVa2267LQAbbrhhYV5L/1fWtWTJEgA++OCDJq/tvvvutTVOMCveIu8z2mOPPQB49dVXAXjmmeINTEcddVSErauOKk4RkYBSVXFuv/32ABx99NGFecuWLQPgzDPPBIp/0aZMmVLRZ26yySYAfPLJJwBcccUVAFx99dUhtFiC8FlAsSJZV2kluXjxYqCY2QknnNDqOvx7Sj/r+uuvB+CFF14I2GKpxDbbbFOY/vDDDwH429/+BsBhhx0GwGmnRdshU7duuW4211svVzv6vY5KqeIUEQkoVRWnN3Xq1ML0v/71L6BYdfi/JEF99tlnADz55JMAfPrpp7U0UQLYYIMNADjrrLMK8y644AIAJk2aBED//v2B4uViAHfffTcAJ598MgDffPNN4bWNN94YKF5G5Cub559/vrDMqaeeGt4/Qppo164dAL169SrMmzdvHgANDQ01fbY/F7HppptW9X5/LqRz585A4+OwlVDFKSISkDacIiIBtbqrbmb3AIcBS51z/fPzugCPANsCHwHHOOdi3bcdPXo0ALvtthsAixYVO7aeNm1axZ/jd9mfeuqpEFuXPEnKtUOHDgCMHDmyMO/SSy8F4M477wSKh2Ja0tyud/fu3QH4/e9/X3M70yLubP1u73777QcUD7tA8QSv5y8HDMrf5eX/z3Tt2rWi9/39738HYP78+VWt16uk4hwLDFln3ijgFedcX+CV/HNJl7Eo16wai7Ktq1YrTufcZDPbdp3ZQ4ED8tPjgEnASOrsxhtvBOD4448vzFu7di0Ajz76KADHHnssAP/8Z3GAuiAVp9exY8fC9Omnn95o/VmQpFy90kuFfKVZK3+Zif+80kvZsirubP3JvscffxyovBoM4re//S0AO+64IwAjRowovPbWW28BsHz58ibvGz58OBD88qN1VXtWvbtzbjGAc26xmW1RbkGNmpcqyjW7KspWuVam7pcjhTlq3sUXXwzArbfeWpjnL3qeMWMGAH/9618B+Mc//lHLqvjWt75VmPZ/3bJUcdYqzFz97a4SvzByHTp0KADPPvssULw0DGCfffYBilVh3759AXj99deraq//vT/wwAML80aNyh2FePnll6v6zEpUe1Z9iZn1AMg/Lg2vSRIj5ZpdyjZE1VaczwA/Aa7LPz4dWosqcM455xSm/V8zfzb8d7/7HQD9+vUrLLP11lsDsHDhwiaf9Z///AeAt99+uz6NTZdYcj3kkEMAGDRoUBSra6siy/b+++8HoH379kCxExaAe++9F4D77rsPgAEDcmOhHXzwwVWty/el6vvTjUqrFaeZPQRMBfqZWYOZnULuyx9sZvOAwfnnkiLKNbuUbf1VclZ9WJmXfhhyWyRCyjW7lG39pfJe9VKnnHJKs/P9iSQo3h/re8Ep5XtRmjlzJlDsyWXNmjWhtlPK85eUvfnmm01eW7dvRn+Ji6THihUrCtOPPfYYULzpYfz48aGs47nnnitM+z5U33nnHaDxZW5h0S2XIiIBpb7iDMuee+4JFHvf8ScsAFauXBlLm9oK34NRaWXi+Yupr7sud0ju888/j65hEorSC9EvvPDCuqxj7NixhemXXnoJgIkTJwKqOEVEEkEVZxmlf6V69uwZY0uyz1f0e+21V2Ge7y/RXxx/1VVXRd8wkTJUcYqIBKQNp4hIQNpVX4e/4+gvf/lLYZ6GB47G+++/X5iudkgEaZv8YI0AL774IgALFiyo2/pUcYqIBJTKitNfRAvl+/orvVf9q6++Aor3s48ZM6bsZ/v7a/v06VNzO6UyfohWf98yNH8xvKSDczV1llWVH/6weFOU//1eurR+/Zio4hQRCSiVFefAgQML0753FN/zkR/fpEePHk3ed9555wHN96Tihxv1n+OXlfrzve37nnMAdtppp1A+2/er6vdAqhkNQIIJOtRuGqniFBEJKJUVZ6kXXngBKI5et2rVKgAuu+yywjK+z81Jkya1+nm+n8Csj3KZJL5H/3r0x+lvXjj00EMBGDJk3THMJGzrr5/brEyYMAGAwYMH121dfvTTH/zgB4V5LZ3DCEsl/XH2NrOJZjbHzGab2Xn5+V3MbIKZzcs/dq57ayU0yjWblGs0KtlVXwNc5Jz7LjAQ+JmZ7YCGG0075ZpNyjUClXRkvBjwo+OtNLM5QE9iHkq2NaUDNfleUvxjS7p16wYUhwQGuOOOO0JuXfySlOvll18OwGuvvVbP1bQJSch15513BmDWrFkAvPrqq4XXSnepw+CH5fjlL39ZmOf72K2nQMc482M17wpMQ8ONZoZyzSblWj8VbzjNrCPwBHC+c25FpZcchDmMbHN8v5l+YHp/OVLpAeJKKk2ve/fuQLEKgmxWnF6Scu3du3dh2g/x6vvhDMr3MP6b3/ym9oalUJy5vvfeewD0798fgHfffbfw2r///e9m3/OnP/2pMF1uVIeWlPZm9uWXXwZ+f1AVXY5kZhuQC+EB55zv617Djaaccs0m5Vp/rVaclvtTdTcwxzl3U8lLsQ0RXDp4/bqXl/jLkerR63OWJDFXf7E6wCWXXALAN998A8ANN9wQ6LP8Me5evXoBMHTo0DCamHhJyNXfcjl37lwANtlkk8Jrvgpd95baESNGFKaHDx9e8bp85x5PPvlkVW2tViW76t8HTgTeMTM/+Phl5AJ4ND/06ELg6Po0UepEuWaTco1AJWfVpwDlDpBouNGUUq7ZpFyjYVH2ZFKPk0Nh87sS/o4kaHzSokYznHMDWl8sXWrN1feOdPDBBxfmPf/880BxcLYlS5YE+sztt98eKJ6YCOve9zKUazaVzVX3qouIBJT6e9XD5qsfic7atWuB4rCuUOwB/qSTTgKKvWBVyr/ff7ZImLSVEBEJSMc4o6VjYdmkXLNJxzhFRMKiDaeISEDacIqIBKQNp4hIQNpwiogEpA2niEhAUV8AvwxYlX9Mm67U3u5twmhIAinXbFKuZUR6HSeAmb2Zxmve0truqKT1+0lru6OS1u+n3u3WrrqISEDacIqIBBTHhrP+o8XXR1rbHZW0fj9pbXdU0vr91LXdkR/jFBFJO+2qi4gEpA2niEhAkW04zWyImc01s/lmNiqq9QZlZr3NbKKZzTGz2WZ2Xn5+FzObYGbz8o+d425rUqQhW+UanHJtYb1RHOM0s3bAB8BgoAGYDgxzzr1X95UHlB9zuodzbqaZdQJmAD8GRgDLnXPX5f8TdXbOjYyxqYmQlmyVazDKtWVRVZx7AvOdcwucc6uBh4FEDnTtnFvsnJuZn14JzAF6kmvvuPxi48iFIynJVrkGplxbUNOGM0Ap3xP4uOR5Q35eopnZtsCuwDSgu3NuMeTCAraIr2X1FXAXLXXZttVcIdu/s1HmWvWGM1/K/x44BNgBGGZmO5RbvJl5ib4Oysw6Ak8A5zvnVsTdnqgEzBVSlm1bzRWy/Tsbea7Ouap+gL2Bl0qeXwpc2tKy5L74tvzzn2q/76h+guRasnzc32vcP4nPtcrf2bi/17h/yuZaS+9IzZXye627kJmdBpwG7FTDurLin3E3oAJBc5V05AoVZKtcGymbay3HOCsq5Z1zY1yul5LDa1iXRCdQri6FPee0Ya1mq1wrU8uGswHoXfK8F7Co3MLOuedrWJdEJ1CukirKNiS1bDinA33NrI+ZbQgcBzwTTrMkRso1u5RtSKo+xumcW2NmZ5M76dMOuMc5Nzu0lkkslGt2KdvwRNo7kplFt7JkmpHFY0fKVblmVNlc1cmHiEhA2nCKiASkDaeISEDacIqIBBT1uOqROeCAAwrTEydOLLvcWWedBcDo0aPr3SQRyQhVnCIiAWnDKSISUOZ21Q8/PHdL/NVXX13R8hdffDEAG2+8MQA33nhjfRomoVtvvfUaPZZek9yuXbuaPvvzzz8HoEOHDjV9jmSTKk4RkYAyV3FuueWWAPTv378wb+XKlQD84he/aPQc4IQTTgDg29/+dlRNlAqY5Try6dKlS9llDjroIACGDs2N6DBr1qzCa9dcc00dWydhGThwIABTp06NtR3+/1ulVHGKiASUmYrz4IMPBmDIkCEALFy4sPDamDFjALjnnnsAWLVqVeG1PfbYI6omSgDt27cHYNmyZRW/59hjjy1Mf/bZZ40eJRlGjswNNLnvvvsCsNlmm8XZnKqp4hQRCUgbThGRgFrdVTeze4DDgKXOuf75eV2AR4BtgY+AY5xzn9avmY0deeSRTeadeuqpAPTt2xeAsWPHFl779a9/3epn+pND+++/PwCTJ0+utZmJlsRcW/Lf//4XgHnz5lW0vL8TrC3eEZa0bK+88srCtP/d9SdvGxoaALjqqquiaEpoKqk4xwJD1pk3CnjFOdcXeCX/XNJlLMo1q8aibOuqoo6M8wO9P1fy12sucIBzbrGZ9QAmOef6VfA5oXSM2lybFy3KDZ3yhz/8AaisyoTiJUr+nvUPPvgAgEGDBtXczmYkqsPbpOVaaqONNgKKF6I//vjjAAwfPjzsVYUhUblCONnW8/d1wYIFAIwfPx6ASy65JIxVha1srtWeVe/unFsMkA9ii3ILarjRVFGu2VVRtsq1MnW/HMk5NwYYA/Xtiv///u//gODHtHxlunz5cgCOOeaYcBuWUfXO9cADDwSKlyMltNLMnHrn+umnucOqf/zjH4HGxz/TpNqz6kvy5T75x6XhNUlipFyzS9mGqNqK8xngJ8B1+cenQ2uRxCkxuT711FMA3HvvvXE1IWsSke2oUblzUg8//HAcqw9NqxWnmT0ETAX6mVmDmZ1C7ssfbGbzgMH555IiyjW7lG39tVpxOueGlXnphyG3RSKkXLNL2dZfZu5Vl2xZvXo1AD/96U9jbomEyZ8cWrFiRcwtqY1uuRQRCSiVFec555xTmD733HND/eztttsOgFtvvbXZ9YlIeHr16gUUb5kGuOuuu4Di7ZhJpIpTRCSgVFact912W2G6uQ4/auH/Ap599tmFeao4o7f++rn/mv67L90DkPTafffdAfjqq68A+M53vgPAFVdcUVjmiy++AIq3Y/rbM5NEFaeISEDacIqIBJTKXXU/SBe0PJiXpNevfvUroDjoWo8ePcou+9ZbbwHw2GOP1b9hEpjvuQyKJ3P9EBrNueGGGwD45ptvALj55pvr2LrqqOIUEQkolRXnHXfcUZj2wwFLtlx77bUArF27Fij2ftWc++67D1DFmVSlAyLefvvtAPzoRz+KqzmhUMUpIhJQKivOfv2KHVdPmDAhxpZIvaxZswYoHt/ylefJJ59cWMYf/5w/f37ErZMgSvcQd9ttt1aX973BP/10cjtdU8UpIhJQKitO3wEEFCsRyaYvv/wSKFYfhx56aOG10pFMJblmzJjR7HQ5vq/OVN9yaWa9zWyimc0xs9lmdl5+fhczm2Bm8/KPnevfXAmLcs0m5RqNSnbV1wAXOee+CwwEfmZmO6DhRtNOuWaTco1AJR0ZLwb86HgrzWwO0BMYChyQX2wcMAkof1WrJEpact18880BuPzyywG46KKL4mpKKiQx17QOyNaSQMc482M17wpMQ8ONZoZyzSblWj8VbzjNrCPwBHC+c26FmVX0vqiGB67VtGnTALjzzjuBttPzeFJzbd++PQD3338/AJdeeikAb7/9dtiryqQk5brVVlsVpn0P8L53pJZsuummALRr167J++NW0eVIZrYBuRAecM6Nz8/WcKMpp1yzSbnWX6sVp+X+VN0NzHHO3VTyUiKGGw3LzJkzAXjwwQeB7FecSc/VV0g777wzoEqzUknMdfr06YXpM844A4Bnn3221fddeOGFQOMOXk4//fSQW1edSnbVvw+cCLxjZv5/72XkAng0P/ToQuDo+jRR6kS5ZpNyjUAlZ9WnAOUOkGi40ZRSrtmkXKORyjuHpO3o3r07AC+//DLQuC9WSYfSk0MbbbRRq8v7/jiPOOIIoLK7jaKme9VFRAJSxSmJ9PXXXwOwyy67ADBr1iwAli1bVtXnnXXWWQA8+uijIbROqnX00blDqzvuuGPZZY477jgAZs+eDSSzn1VVnCIiAaW+4hw0aFConzdp0iSgeDmMxMO53LXX7733HgADBgwAiuMLBeUvqJfoPffcc4XpDh06AMVhgjfbbDMA9t1338IyU6ZMAWD06NFAMvvlVMUpIhKQ+b/skawswbdcRmSGc25A3I0Im3JVrtUaOHAgAFOnTi3M23vvvQF4/fXX67361pTNVRWniEhA2nCKiASU+pNDIpJefnc8bSdjVXGKiAQUdcW5DFiVf0ybrtTe7m3CaEgCKddsUq5lRHpWHcDM3kzjGci0tjsqaf1+0truqKT1+6l3u7WrLiISkDacIiIBxbHhHBPDOsOQ1nZHJa3fT1rbHZW0fj91bXfkxzhFRNJOu+oiIgFpwykiElBkG04zG2Jmc81svpmNimq9QZlZbzObaGZzzGy2mZ2Xn9/FzCaY2bz8Y+e425oUachWuQanXFtYbxTHOM2sHfABMBhoAKYDw5xz79V95QHlx5zu4ZybaWadgBnAj4ERwHLn3HX5/0SdnXMjY2xqIqQlW+UajHJtWVQV557AfOfcAufcauBhYGhE6w7EObfYOTczP70SmAP0JNfecfnFxpELR1KSrXINTLm2oKYNZ4BSvifwccnzhvy8RDOzbYFdgWlAd+fcYsiFBWwRX8vqK+AuWuqybau5QrZ/Z6PMteoNZ76U/z1wCLADMMzMdii3eDPzEn0dlJl1BJ4AznfOrYi7PVEJmCukLNu2mitk+3c28lydc1X9AHsDL5U8vxS4tKVlyX3xbfnnP9V+31H9BMm1ZPm4v9e4fxKfa5W/s3F/r3H/lM21lt6Rmivl91p3ITM7DTgN2KmGdWXFP+NuQAWC5irpyBUqyFa5NlI211qOcVZUyjvnxrhcLyWH17AuiU6gXF0Ke85pw1rNVrlWppYNZwPQu+R5L2BRuYWdc8/XsC6JTqBcJVWUbUhq2XBOB/qaWR8z2xA4DngmnGZJjJRrdinbkFR9jNM5t8bMziZ30qcdcI9zbnZoLZNYKNfsUrbh0bjq0dL429mkXLOpbK4a5VJEYrfhhhsWpr/++msAVq9e3WS5Tp06lX0tSuodSUQkIFWcIhKbzTffHIBly4oDUn7yyScAdO3atcny/tBi3OOwq+IUEQlIFaeIxMZXmh999FFhXp8+fcouX7pcnFRxiogEpA2niEhA2lUXkdi1tHtezXL1popTRCSgzFWcr732GgBXXXVVYd7LL78cV3OkSuXuaNt1110L02+//Xarn9OxY0cAttpqKwA+/PDDwmv/+9//ammihGDq1KlxN6EqqjhFRALKTMU5efJkAPbYY49Wl7322msL06eeemrgdU2fPr0wfeihhwZ+v1TvrbfeKkxXchH04MGDARg/fjzQOPvLLrss5NZJUPvss0/cTaiKKk4RkYC04RQRCajVXXUzuwc4DFjqnOufn9cFeATYFvgIOMY592n9mlme30Xfa6/c0CkbbLBBq+/xJwyg+fthWzNo0KDC9IIFCxq99umnua9h9913D/y5UUp6rpW49dZbATjnnHOavNahQwcADjroIKCY2QMPPFBYJqu76lnINukqqTjHAkPWmTcKeMU51xd4Jf9c0mUsyjWrxqJs66rVitM5Nzk/0HupocAB+elxwCRgZIjtqljv3rkhVEr782tN6aUuvmq5++67q1r/dtttB8Djjz/eqD2ll8p873vfq+qz6ynpufoTP2eccQYAN910EwDdunUrLNPSZWbt2rUDoHv37gBMmjQJgK+++ir0tiZN0rPNgmrPqnd3zi0GcM4tNrMtyi2o4UZTRblmV0XZKtfK1P1yJOfcGGAMhNcV/7PPPluY3nLLLRu9duaZZwKNLxlqyeLFiwGYNWtWVW2ZP38+APvtt19V70+reuTaHP/97r///gCsWrWq8NrMmTMB6N+/PwDvvvtuk/f36tULKO4ZSMuiyrVaZ599NgC33XZbrO2o9qz6EjPrAZB/XBpekyRGyjW7lG2Iqq04nwF+AlyXf3w6tBZV4JZbbilM+wveN9poI6B4bNGf3a43XwFNmTIlkvXVWay5NufPf/5z2dceeughAI488kig+YrT//948MEHgeIxzzYocdlWw5+TSHzFaWYPAVOBfmbWYGankPvyB5vZPGBw/rmkiHLNLmVbf5WcVR9W5qUfhtwWiZByzS5lW3+pvFf92GOPLUxvsskmAPz85z8HiicTREpV0oeBSKV0y6WISECpqjjHjBkDwAknnFCY5/vd9BewL1++vNXPiXtoUam/cv156rKkZPInfaD5W2iTRhWniEhAqao458yZAzTunMFXmitWrGj1/ccddxwAS5cWL2F75ZVXwmyihGzgwIEAHHXUUU1e8726z507N9I2SXjuvfdeAEaMGFGY5zvhOfnkkxstUzovbqo4RUQC0oZTRCQgK3cQvS4ri/neV1/yz5gxozAv4jsQZjjnBkS5wijUI9ett94agDfeeANo+Y6fTz75pNFjvk0A9O3bF4Arr7wSgGuuuaawTIiDtSnXKvntT3OHYoYNy12O6u8My7ep3k0qVTZXVZwiIgGl6uTQBRdcAMDtt99emNcW+ldsi/yNDL5C9BVjqaFDhwJw/fXXN3nNVzKLFi0CYMmSJYCGBE6Kl156qdHzJ554oskyvo/bJFLFKSISUKoqTt8L+H333VeYV0nFeeCBBwKw/fbbA42PcUoyLVy4EGhaaZ577rmFaX/rbb9+/cp+zuGHHw7A8OHDAXj44YcLr3322WehtFWC82NB7b333jG3pDqqOEVEAqpklMvewH3AlsBaYIxz7pY0jZrnx63x49DMmzcvzuYkQtpyvfrqq4HGNz8MGFD5iWx/ZnbkyOIwO1msONOW6+uvvx53E6pSScW5BrjIOfddYCDwMzPbAY2al3bKNZuUawRa3XA65xY752bmp1cCc4Ce5EbNG5dfbBzw43o1UsKnXLNJuUYj0Mmh/JCjuwLTCDAiYlKMHj0aaHopRFuXxFz9Be/+siJ/EvDLL78sLFM6BLM0lcRcK/Hhhx8C0KdPn0CvRaniDaeZdQSeAM53zq2o9Ap+DTeabMo1m5RrfVW04TSzDciF8IBzbnx+9hIz65H/61V21Ly4hhv1Q8YCdO3aNarVpkqSc/X9Zt58881A8VK0N998M9DnfPzxx0DbOiGY5Fw9/zu5bNmyJq/5y8tKb6H1/AlB/764frcrGazNgLuBOc65m0pe8qPmQYpHzWurlGs2KddoVFJxfh84EXjHzPxBpcvIjZL3aH4EvYXA0fVpYjC+UvG9xQP06NEDiG7I4JRIdK7+Fsnjjz++ps/xFWpLwwxnTKJz9Xw16ftULbV69epW37f55psD8PXXXzdZpn379mE0sUWVjHI5BSh3gESj5qWUcs0m5RoN3TkkIhJQqu5V97bZZpvCdKdOnRq99sgjjwCNdwF8d/svvvhiBK2TJNIAfcnU0m55S+LOUxWniEhAqao4/SUI06dPL8xbb73G235///ERRxxRmDdx4sQIWidJFuVIB5J9qjhFRAJKVcXZrVs3oPFFs/6yBO+QQw4B0tvritRH3MfEJFtUcYqIBJSqitPTLZRSqcGDBwPw/vvvA8We5UVqoYpTRCQgbThFRAKyKC/TiLJ3pIQqO8B9milX5ZpRZXNVxSkiElDUJ4eWAavyj2nTldrbvU3ri6SScs0m5VpGpLvqAGb2Zhp3a9La7qik9ftJa7ujktbvp97t1q66iEhA2nCKiAQUx4ZzTOuLJFJa2x2VtH4/aW13VNL6/dS13ZEf4xQRSTvtqouIBBTZhtPMhpjZXDObb2ajolpvUGbW28wmmtkcM5ttZufl53cxswlmNi//2DnutiZFGrJVrsEp1xbWG8Wuupm1Az4ABgMNwHRgmHPuvbqvPKD8mNM9nHMzzawTMAP4MTACWO6cuy7/n6izc25kjE1NhLRkq1yDUa4ti6ri3BOY75xb4JxbDTwMDI1o3YE45xY752bmp1cCc4Ce5No7Lr/YOHLhSEqyVa6BKdcWRLXh7Al8XPK8IT8v0cxsW2BXYBrQ3Tm3GHJhAVvE17JESV22yrUiyrUFUW04m+t+O9Gn882sI/AEcL5zbkXc7UmwVGWrXCumXFsQ1YazAehd8rwXsCiidQdmZhuQC+EB59z4/Owl+eMp/rjK0rjalzCpyVa5BqJcWxDVhqWPGmoAAACmSURBVHM60NfM+pjZhsBxwDMRrTsQyw1Oczcwxzl3U8lLzwA/yU//BHg66rYlVCqyVa6BKdeW1hvVBfBmdijwO6AdcI9z7teRrDggM9sX+CvwDrA2P/sycsdNHgW2BhYCRzvnlsfSyIRJQ7bKNTjl2sJ6deeQiEgwunNIRCQgbThFRALShlNEJCBtOEVEAtKGU0QkIG04RUQC0oZTRCQgbThFRAL6f218j0fB+yiwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "datagen = ImageDataGenerator(featurewise_std_normalization=True)\n",
    "# fit parameters from data\n",
    "datagen.fit(Xtrain)\n",
    "# configure batch size and retrieve one batch of images\n",
    "for X_batch, y_batch in datagen.flow(Xtrain, ytrain, batch_size=9):\n",
    "    # create a grid of 3x3 images\n",
    "    for i in range(0, 9):\n",
    "        plt.subplot(330 + 1 + i)\n",
    "        plt.imshow(X_batch[i].reshape(28, 28), cmap=plt.get_cmap('gray'))\n",
    "    # show the plot\n",
    "    plt.show()\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU4AAAD7CAYAAAAFI30bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZQU1dnH8e8jghsgILIIBFSIiEviEvHV1wQXPL4JiktcOGr0qMcFUYwbbnHBNRqXuEvEQCKCGGJccIEgaDxuCBoNEIGoLIIsEsANEb3vH923qmbomenq6a7u6vl9zpnTPVVF16Wf6dvPvXXrXnPOISIi+duk3AUQEUkbVZwiIjGp4hQRiUkVp4hITKo4RURiUsUpIhJToypOMzvMzD4ws/lmdlmxCiXlpbhWL8W2OKzQcZxm1gyYC/QHFgPTgUHOudnFK54kTXGtXopt8WzaiH+7DzDfOfchgJmNAwYCdQbBzJr6aPuVzrlty12IBiiu8aUhrhAztopr3XFtTFO9C7Ao8vvi7Dap24JyFyAPimt8aYgrKLZx1RnXxmSclmPbRt9QZnYmcGYjziPJqui4brJJ+F3fsWNHAHr37g3AgAEDgn0XXXRRsgVLhwZjq89rfhpTcS4GukV+7wosqX2Qc24EMAKU+qeE4lq9Goyt4pqfxjTVpwO9zGx7M2sBnAA8XZxiSRkprtVLsS2SgjNO59wGMxsCvAg0Ax5xzs0qWsmkLCo1rptumvlTPeCAA4Jto0ePBmDOnDkAzJpV9mJWtEqNbRo1pqmOc+454LkilUUqhOJavRTb4mhUxVnNohchWrZsCcBmm20GQLNmzYJ9fhzssmXLEixd07PVVlsBcOKJJwbbJkyYAMDKlSsBmDFjRvIFkyZJt1yKiMSkjDPLZ5itW7cGoEePHsG+UaNGAbD77rsDYZYJ8NFHHwHQs2fPBErZ9DRv3hyAvn37AvDOO+8E+8aMGQPArrvuCsCrr76acOmkXHyrz38Wv//++0TPr4xTRCSmJpVx+qyyXbt2wbYOHToA0LZtWwCuvPJKoGYGuWbNGgDeeustoOa325IlGw1xlCLyMevcuTMAq1atCvatXr0aUKbZlPjrDddffz0AU6ZMAeCFF14IjtmwYUPJy6GMU0QkJlWcIiIxVW1T3Sy8Ldc3x/fcc08Ajj/++GCfH94yd+5cIBxEfdxxxwXHvPvuu6UtrNTJd/5//fXXAKxfv76cxZECbL755sFzf4HVd3ctWJCZR2PFihV1/vvoZ7l9+/ZAeHHo/vvvB+Ccc84Jjpk4cWIxil0vZZwiIjEVPJFxQSdLYNIA/+30gx/8INh28cUXA+G3kr9FD+CNN94A4NFHHwXg5ZdfLmXxZjjn9i7lCcohibj6i0TRv9ck/3YboLjm0KJFC6BmNnj++ecDsGhRZnY738K77rrrgmOWL1+e83UATj/9dGDjC72DBw8OjvEXEougzrgq4xQRialq+jh9ptmlS2Ze1mHDhgX7+vXrB8Dvfvc7AKZNmxbsiw5jkMqV9ABnaTx/i/Kll14abLviiiuAsG/zmmuuAaBXr17BMbUzzly++eYbAO666y4ATj311MYXOAZlnCIiManiFBGJqcGmupk9AgwAljvnds1uawc8DvQAPgaOc879t3TFbNi222bWVBo+fDgAP/nJT4J9vtn+zDPPJF+wCpWWuPphJz/96U8B+O6774J9r7zySlnKVOkqJbZ+CNmQIUOCbf4z6Ocg8MMA/R17EA4X/Pzzz4HcFwF915zvwonOZpaEfM42Cjis1rbLgCnOuV7AlOzvki6jUFyr1SgU25JqMON0zr1iZj1qbR4I9Ms+Hw1MA4ZRBv6bp1WrVgDsv//+QM1vub///e/JF6zCVXpca/PDy6IZp+RWKbH194xHW3p+m4/jbbfdBsAf//jH4Bh/USlXxulbIP7f+4wzOkduEgq9qt7RObcUwDm31Mw61HWgVs1LFcW1euUVW8U1PyUfjpTUqnn+G+iLL74AwllTsmUo1WmbrKTjOm7cOABGjBgR7HvooYeAcHZ4abxSxDXXbEX+M7lw4UIg7A+N8q1JP5wQ4NZbbwVg6NChQJhxRm/LTEKhParLzKwzQPax4YFXkgaKa/VSbIuo0IzzaeAU4Jbs41NFK1FM/pvLz4s5aNAgQAOmC1Qxca2tU6dOQHiDA2gezpgqMrb5XBWP9muPHz8egMcffzzvf18KDZ7NzMYCrwM7mdliMzudzJvf38zmAf2zv0uKKK7VS7EtvXyuqg+qY9fBRS6LJEhxrV6KbelVzb3qfp7GefPmlbkkUkxt2rQB4OSTTwZqNs/vvvvuspRJiqe+izu+G+6//w3H6e+1115AOI/uww8/DFRgU11ERGqqmozTz8H37LPPAvCzn/0s2OeHQ6xbty75gkmj+Ixzhx12AOCMM84I9ukCYHrtsssuQHgrbdeuXYN9tbPP6AoMfsZ3P+D9qKOOAsJB80lRxikiElPVZJx+yIKfEXrx4sXBvrFjxwI1Z6KWdBgzZgwARx55JKAsM+123HFHAO68804ADjzwQAA23TSsiuobzO77Mo899lgAevToAcBNN91U9LLWRxmniEhMqjhFRGKqmqb66tWrAbjxxhsBOOWUU4J9/r5WSZ+OHTsCsGrVqjKXRArVunXr4PnIkSOBcOajo48+GoAbbrghOMbPS/Dtt98CNZvu3bp1A6B3794AXHLJJQDcc889JSl7XZRxiojEVDUZZ9u2bYFwmdFRo0YF+3RBIb38IOhSzn7jl5/1N1FIcUWX9/XDiPxyvp988glQs1V40EEHAfUPivef6RkzZgDJx04Zp4hITKnPOP03mB+e4AfI3nHHHcExueYDlHTw8Yw7p6oftuKzltq/R7f51QL8EBnQTPOl4m9CWbZsGRBmjtFWYffu3YFwSGE09meddRYAW265JQCvvfbaRsckQRmniEhMqc84/cBZ/03kb+HKNaN0Lj4Dqb1qnlSGnj17AmF2GM0E/ZVUn6Hkyia9XP1l/qpt3759Abj33nuDfco4S2PzzTcHYLvttgPgm2++AcIMFMK45MoiV65cCYQTvMyePRuAyZMnB8ck8RnOZz7ObmY21czmmNksMxua3d7OzCab2bzsY9uSl1aKRnGtToprMvJpqm8ALnLO7QzsC5xrZn3QcqNpp7hWJ8U1AflMZLwU8KvjfW5mc4AuVMhSsrfckpnI2i/i5Bdrq0+02TZw4EAgbDL4pYR9c6FaVXpcPb8Uyi9+8QugZux8cy96n7NX+6JD7eVkAa688koAzj33XCD8G0izSoxr9H396quvgLCp7eN53333BcesWLGizteaMGECEC4D7uOa9GJtsfo4s2s17wG8iZYbrRqKa3VSXEsn74rTzFoCE4ALnHNr863hS72MrM8YfeZZX6bolxkdNiz8ov3hD38IwIMPPgjA1KlTG3ydalKpcfX84my5Msbag6G1DHSokuLqb68EOO+884BwRv/+/fsX9JpJ3BhRn7yGI5lZczJBGOOc+2t2s5YbTTnFtToprqXXYMZpmSp9JDDHOXdHZFdFLDe6xRZbAGHfZK7lQv0geT+buJ84AsJJB/ycndXQz5WPSo+rl++wMsmo9Lj6NcGGDx8OwMSJEwFYtGhRcEw+w4lqH1OJfZz7AycD75uZn8P+CjIBGJ9denQhcGxpiiglorhWJ8U1AflcVX8VqKs613KjKaW4VifFNRmpv3PIp+y77rprncdMmzYNgJ///OcAfPbZZ8E+3zTXzDgipecv6viLr2+++WaN7XFfp6IvDomISCj1GWefPn2A8BvID1uJ3mvsv910/7FIZSl0CJmf8axcmacyThGRmCzJQcOlHCidEjOcc3uXuxDFprgqrlWqzrgq4xQRiUkVp4hITKo4RURiUsUpIhKTKk4RkZhUcYqIxJT0APiVwJfZx7RpT+PL3b0YBalAimt1UlzrkOg4TgAzezuNY97SWu6kpPX9SWu5k5LW96fU5VZTXUQkJlWcIiIxlaPiHFGGcxZDWsudlLS+P2ktd1LS+v6UtNyJ93GKiKSdmuoiIjGp4hQRiSmxitPMDjOzD8xsvpldltR54zKzbmY21czmmNksMxua3d7OzCab2bzsY9tyl7VSpCG2imt8ims9502ij9PMmgFzgf7AYmA6MMg5N7vkJ48pu+Z0Z+fcTDNrBcwAjgROBVY5527J/hG1dc4NK2NRK0JaYqu4xqO41i+pjHMfYL5z7kPn3HpgHDAwoXPH4pxb6pybmX3+OTAH6EKmvKOzh40mExxJSWwV19gU13o0quKMkcp3ARZFfl+c3VbRzKwHsAfwJtDRObcUMsECOpSvZKUVs4mWutg21bhCdX9mk4xrwRVnNpW/D/g/oA8wyMz61HV4jm0VPQ7KzFoCE4ALnHNry12epMSMK6Qstk01rlDdn9nE4+qcK+gH+B/gxcjvlwOX13csmTe+Kf+sKPT9TuonTlwjx5f7fS33T8XHtcDPbLnf13L/1BnXxsyOlCuV71v7IDM7EzgT2K0R56oWC8pdgDzEjaukI66QR2wV1xrqjGtj+jjzSuWdcyNcZpaSoxpxLklOrLi6FM6c04Q1GFvFNT+NqTgXA90iv3cFltR1sHPuuUacS5ITK66SKoptkTSm4pwO9DKz7c2sBXAC8HRxiiVlpLhWL8W2SAru43TObTCzIWQu+jQDHnHOzSpayaQsFNfqpdgWT6KzI5lZcierTDOqse9IcVVcq1SdcdUkHyIiManiFBGJSRWniEhMqjhFRGJKel31imAWjgNu1qwZAJtuummN3zfZJPxOWbduHQDffvttUkUUkQqmjFNEJCZVnCIiMTWJpvoWW2wBwGmnnQbAvffe2+C/Wb16dfB8yJAhADz3XOauUd+ch7BJv2HDBgBWrVpVhBJLbVtvvTUAN998c7DtqKMy0x/4LpTHHnss2Peb3/ymxj6RYlLGKSISU9VknK1atQLgtttuA2D//fcP9vkLP23bZtZrit4t9f3339d4/O6772r8G4BrrrkGgIsuugiA5s2bB/s222wzAN577z0ABg8eDMDy5csb/5+SIKPv1asXAD/+8Y+DfX37ZmZE23LLLQEYNWpUsG/HHXcE4N///ncSxZQctttuOwCuvvrqYNsTTzwBwJQpU0p+fn8RuHXr1sG2jh07AuFn2H/e4/6dKOMUEYkp9RnnVlttBcCdd94JwKGHHgrAuHHjgmNefPFFYOPsMspnofXta9GiBQADBgwI9p166qkA7LXXXgDcfvvtAJx88smF/HekFv+en3jiiUCY0QMsXLgQCLP+iRMnJlw6yaV9+/YA/Pa3vwVg3333DfYVO0Y+q/T1AITXNHwrdODAcI25c889t8a/W7QoM69zv379Yp1XGaeISEyqOEVEYmqwqW5mjwADgOXOuV2z29oBjwM9gI+B45xz/y1dMWvyFwMA7rvvPgD69+8PwJ/+9CcARo4cGRzzn//8B6h5UagQ/kKFT+8BXnnllRqvvX79+kadIymVGNdc/Ht+wAEHAHDhhRdudIx/73PdEdYUlTu2/sJLnz6ZBTT//Oc/B/vefffdopzDd8/4C75Dhw4N9vmuAf834I8B2GabbYCwab9s2bKCzp9PxjkKOKzWtsuAKc65XsCU7O+SLqNQXKvVKBTbkmow43TOvZJd6D1qINAv+3w0MA0YVsRy1SuaTRxzzDEAPPjgg0CYafosExqfaXr+wtH8+fODbdHnaVKJcc3Fx85nCNGhJWvWrAHCuESz/bfeeqvGv2tKyh1bfzOIN3369OD5fvvtB8CCBZkFJN94441Yr92yZUsArrrqKgC6dcssobTPPvsEx3Tq1AkI/x7eeeedYN+0adMA2HvvzPzE119/fazze4VeVe/onFsK4JxbamYd6jpQy42miuJavfKKreKan5IPR3LOjQBGQPGm4o8OGfID1ceMGQMUrz9T6leKuObisxef2fvbXwFuvPFGIBzEPHXq1GCfH3Yi8RQaV9/nCHD00UcD8I9//AOAPffcM9h3yCGHAOFwwbgZp/+8H3fccQB07twZgL/85S/BMbNmZZZR8vVEdKYz31r1wxeff/75WOf3Cr2qvszMOgNkH3WbTHVQXKuXYltEhWacTwOnALdkH58qWonyEO3LeuqpzKk/++wzQJlmI5U1rrn4jNNP2hEdzOz5jOKII44ItnXt2jWB0qVKSWMbvQ3Z9z/66w5+MhaAL7/8EghvXojLT9riH/3EOz6DBJg5cyYQ/l1EM97ddtsNgPHjxxd0fq/BjNPMxgKvAzuZ2WIzO53Mm9/fzOYB/bO/S4oortVLsS29fK6qD6pj18FFLoskSHGtXopt6aXyXvXoHIunn346AF999VW5iiMl5LtefJM9Osjdq28OAklGdAiS7ybxN6V8/fXXwb677roLKHx2JP/Z9xcEf//73wNh8zzK/z28/fbbwbbo88bQLZciIjGlMuOM8oNd/W2QabnlUeLJdVtl7X3RC4O77LJLMgUTIPdM+9deey0Ac+fODbb5i7iFfk59ZuuHJUWHGiVJGaeISEypzzh9n4WfBOJf//pXOYsjJVJfxln7mIaOk+LzfY5Rn376KQBLliwp2nl8pun7TcvVr62MU0QkJlWcIiIxpb6p7mdLiS6uJtUrV3M818UhNdXLx18o8p/JaCwKubMvOv+uv7f9vPPOA2rOvJQkZZwiIjGlPk3zndJjx44F4JtvvgFqztk5evRoILyfNVdHtqRDNGPxQ1F8PHNlM7WzUim92hlnY0U/y355Xz8Xq/+8J00Zp4hITKnPOP3MK3fccQcAf/vb3wD40Y9+FBzjlyutnaFIevhhJ/X1YyqrrAx+cPtFF10EhMsEA8yePRvIbxiRzy5vvfXWYJvv7yz3NQ1lnCIiMaU+45w8eTIQ9nn4AfCTJk0KjvETgCjTTL98M85y3YoncO+99wIwaFBmkqbtt98+2PfBBx8A+WWcfr0oP1kIhKvaLl26FChfKyOf+Ti7mdlUM5tjZrPMbGh2ezszm2xm87KPbRt6Lakcimt1UlyTkc/X8gbgIufczsC+wLlm1gctN5p2imt1UlwTkM9ExksBvzre52Y2B+hChSwle8kllwAwceJEIGyqz5s3Lzimvia6b+41tQtHlR7XfNRuqudaxK+pDUeqhLj6OTf9wmzRCzn5dKF06JBZgPPggzPzLvuFGCG8CLx8eXmXTIrVx5ldq3kP4E203GjVUFyrk+JaOnlXnGbWEpgAXOCcW5vvLW2lXkZ28ODBAPzqV78CwuWB68sc/ZKiAMceeywQLm/qB8uvWLEiOKaas5VKjWs95w2e15dx+kX8/Lyca9euTaB0laOccfVzbr7++utAOGcuQL9+/YDwc+oHy++www7BMd27dwdg9913B2D48OHBvujnspzyuvRoZs3JBGGMc+6v2c1abjTlFNfqpLiWXoMZp2W+qkYCc5xzd0R2VcRSsi+++CIQ9nnk00fZrVu34LnvI33ttdeAxi8bmhaVHte61DfkaPXq1cFzP19jU5vso5LietNNNwEwdOjQYJtvIb700ktA2BI47rjjgmP++c9/AnDBBRcAlZNlRuXTVN8fOBl438zezW67gkwAxmeXHl0IHFuaIkqJKK7VSXFNQD5X1V8F6vra1nKjKaW4VifFNRmpv3Po/PPPB6BXr14A7LfffgAsWLBgo2N9p/Nuu+0WbPNNBf86y5YtK11hpWD5zLn5wAMP5Hwu5bFy5UognJUMoE2bNkB4V9HWW28NwMsvvxwcc/nllwPFXXKj2HRfmohITKnPOP196Oeccw4Ahx9+OACPPvpocIzPUk488UQA1q1bF+xbvHgxUL5FnySeaJya2oWftFq1alXw/KqrrgLghhtuAMLlfv3FvNrPK5UyThGRmFKfcXp+6MLmm28OhMMeIJzD79lnnwXgjDPOCPZV4lAH2Vh9s7tLevhZzPxjWinjFBGJyZK8nTCJW/NatGjhzxVs84Pi61ubJiEznHN7l+vkpZJEXH08o3GtoH5pxbU61RlXZZwiIjGp4hQRialqLg55fqEoqS65BsCLlIsyThGRmJLOOFcCX2Yf06Y9jS9392IUpAIprtVJca1DolfVAczs7TRegUxruZOS1vcnreVOSlrfn1KXW011EZGYVHGKiMRUjopzRBnOWQxpLXdS0vr+pLXcSUnr+1PScifexykiknZqqouIxKSKU0QkpsQqTjM7zMw+MLP5ZnZZUueNy8y6mdlUM5tjZrPMbGh2ezszm2xm87KPbctd1kqRhtgqrvEprvWcN4k+TjNrBswF+gOLgenAIOfc7JKfPKbsmtOdnXMzzawVMAM4EjgVWOWcuyX7R9TWOTesjEWtCGmJreIaj+Jav6Qyzn2A+c65D51z64FxwMCEzh2Lc26pc25m9vnnwBygC5nyjs4eNppMcCQlsVVcY1Nc69GoijNGKt8FWBT5fXF2W0Uzsx7AHsCbQEfn3FLIBAvoUL6SlVbMJlrqYttU4wrV/ZlNMq4FV5zZVP4+4P+APsAgM+tT1+E5tlX0OCgzawlMAC5wzq0td3mSEjOukLLYNtW4QnV/ZhOPq3OuoB/gf4AXI79fDlxe37Fk3vim/LOi0Pc7qZ84cY0cX+73tdw/FR/XAj+z5X5fy/1TZ1wbMztSrlS+b+2DzOxM4Exgt0acq1osKHcB8hA3rpKOuEIesVVca6gzro3p48wrlXfOjXCZWUqOasS5JDmx4upSOHNOE9ZgbBXX/DSm4lwMdIv83hVYUtfBzrnnGnEuSU6suEqqKLZF0piKczrQy8y2N7MWwAnA08UplpSR4lq9FNsiKbiP0zm3wcyGkLno0wx4xDk3q2glk7JQXKuXYls8VbeueoXT+tvVSXGtTlpXXUSkWFRxiojEpIpTRCQmVZwiIjElva56RTALxwFvsknmu6NZs2YAfPfddzUeRaTy3HPPPQAMGTIEgC5dwvlHliwp/dBUZZwiIjGp4hQRialqm+pbb7118PzJJ58E4MADD9zoOD+O9csvvwTgggsuAGDkyJGlLqLE0Lx58xqPABs2bKjx+P333ydfMInFd5N16tSpxvalS5fGep2PP/4YgC+++AIIP8dJUcYpIhJT1WSc/iJPz549Abj00kuDfX7bE088AcBzz4XzjbRu3RqAs88+u8br+EdQJlNOLVq0AOCYY44B4O677w72tW/fvsax0azDZyQnnHACADNmzAB00a8cttxyy+D58ccfD8BBBx0EhJ+z6Of1k08+afA1b7/9dgAOP/xwQBmniEjFS33G6b+xdt55ZyDMMPw3G8C0adMAGD06s3bTxIkTg32tWrUCoE2bNgBss802ABxwwAHBMS+//HIpii558HE85ZRTgLCFAGFLwP8NRIeZbbvttgBcdllmWZ37778fgFdeeSU4Zv369aUqtkTccsstwfMBAwYA4TUEH8PBgwdvdPznn3+eVBFjU8YpIhKTKk4RkZgabKqb2SPAAGC5c27X7LZ2wONAD+Bj4Djn3H9LV8y6+Tt+Dj30UAAuvPBCABYvXhwcc/PNNwPw2muvbfTvfXPg1ltvBcJO55122ik4phqb6pUeV9/83nvvzKxe++23H1Czee27XNasWQPUbKq3bNkSgJ/+9KcAvPTSSwC8+uqrpSx2Rai02PpuFoCBAzNLs/vuM8/HCeChhx4C4KSTTgLyuzhbiReHRgGH1dp2GTDFOdcLmJL9XdJlFIprtRqFYltSDWaczrlXsgu9Rw0E+mWfjwamAcOKWK681f6mmTt3LhB+WwF8+OGHDb6O/1bzj9HspRpVelx33XVXADp06ACEA51nz54dHHPFFVcAYcyjfwtdu3YF4OGHHwbCDLZt27bBMZ9++ulG/64aVFps161bFzyPXpyLim6fMGFC3q992mmnAfDZZ58VWLrCFHpVvaNzbimAc26pmXWo60AtN5oqimv1yiu2imt+Sj4cyTk3AhgBxZuK3/drApx66qlAmElce+21AHzwwQfBMf6WvAbKWeP36AB42Vgp4ho1duxYADp27AjAlClTgDDLBFi4cCGQuw/M38J31llnAXDeeecBNYe93HDDDQB88803RS17mpUirj77h/z6K/1MR/Ud62+E8NlpPq3KYiq0dlhmZp0Bso/Li1ckKSPFtXoptkVUaMb5NHAKcEv28amilSgP0exw000z/wXfz/XMM88A8W+TrH18tfdx1qGscY3q0aMHAI899hgQ3rzw0UcfBcfUF2PfyliwYAEAW221FVCzj7Pa+jYbULbYfvvtt7GO9yMnVq5cCYQ3M0DYYvCf97r6TEutwYzTzMYCrwM7mdliMzudzJvf38zmAf2zv0uKKK7VS7EtvXyuqg+qY9fBRS6LJEhxrV6Kbeml8l716DT548aNA8LhKoXOZKSLQ5XFd8G8+eabQHjzQtz4+rj62ZF23HHHYN/RRx8NwPjx4wt6bSkt311WX5dKubpbVDuIiMSUqozTfwNF59P08/g9//zzjXrt2tmGMs7y8jO9F2t291GjRgE1b4x44IEHgDDjlMryhz/8AajZwvSzl9W+ZTNpqh1ERGJKVcbp+zP69OkTbIuuLVRMTXQ4UlnlyvKL1e/oh8RE+8T83476NiuTv0HhmmuuCbb5iV3KNQzJU8YpIhKTKk4RkZhS1VTPxd+3ns/QhXz4Zlv0dXwTUk260vJDkCBcVK1Yw018DKPniHtHiyTLL/LWt2/fYNsll1wChHMP+Lly33vvvUTLpoxTRCSmVGac0SVefaZZrIwz179XxpkMPwQJSpdxRmfWymfWLCm/XH8D/gLx+++/n3RxAGWcIiKxpTLjjGZ+uZaGLdZrexqalIxo/6OPQ7R10Rg+htEhT+rjrEw+Rn5O1VzrRJ1zzjlA2EqJfkaTuA1TGaeISEz5TCvXzcymmtkcM5tlZkOz29uZ2WQzm5d9bNvQaxXLd999F/xssskmRb090jmXc8KParsFsxLj2rx58+DHx/f7778vSt+yj2H0Z8OGDVXXz1mJcY2rd+/e9O7dm+22247tttsu5zE+hpMmTWLSpEm0b98++ElCPrXBBuAi59zOwHSANXgAAAWtSURBVL7AuWbWB62al3aKa3VSXBPQYMXpnFvqnJuZff45MAfoQmbVvNHZw0YDR5aqkFJ8imt1UlyTEeviUHbJ0T2AN4mxImKx+Tk4IVxKoWfPngBsv/32AEydOjU4Js5iXPUNgK9WlRLXXAPgi6UpDoCvlLjmI/oZu/jiiwH49a9/DYSD3KP859NfODrkkEOCfX6hv1LKu+I0s5bABOAC59zafK80a7nRyqa4VifFtbTyqjjNrDmZIIxxzv01u3mZmXXOfnvVuWpeKZYbHTJkSPC8VatWQDibd//+/QGYPn16cEycjHP58sx/oykMQaq0uOYajlSsmw58BusX+QJ48skni/LalabS4pqPaJxPO+20GvuuvvrqjY73GaefOenyyy8vYek2ls9VdQNGAnOcc3dEdvlV86DMKyJKfIprdVJck2ENDRY1s/8F/gG8D/ivhSvI9JuMB34ALASOdc6tauC1iv4Ndt111wHQsmVLAEaMGAHA/Pnzg2Pi9Je1adMGqJn9fPbZZ0BRBtbOcM7t3dgXKYZKjKtfEhjCFsPZZ58NwIQJExr12r4F0a5du2Cbn008moUWSHEtkO/bHDZsWLDt/vvvB+D6668H4Pzzz2/wdUo0AL7OuOazyuWrQF3tVq2al1KKa3VSXJNR3ZeLRURKIJX3qkd16tQJCGe6WbJkCVD4cJbVq1cDNVN/P6NOtd1lUmmiMxetWbMGCJty/p7kQocQ+bkdjzjiiGCbf37UUUcV9JrSeP5z5i/qAowenRluesIJJwD5NdWTXiZYGaeISEypzzjfeOMNAHbYYQcADj44040zadKk4Jivvvoq9ut27949eH7ggQcC8Pjjjxf8etKwFStWBM+HDx8OhPMu+uEqhV4k8hf7fAsFwpsmpHx8y/CXv/xlsO3tt98Gks8i41DGKSISU+ozzkcffRSAk046CQhv05o5c2ZwzKJFi4B432C77LJL8NwPeXrhhRcAZZylsnbt2uD5008/DcCAAQMAWL9+faNeO9ctl5rRv3KsWhWOjPKtx0qmjFNEJKYGB8AX9WQlHFDrMwmfWTQ2QymRihkoXUxJ3ppXKD/w3a+OCOFV9b322quxL6+4Vqc646qMU0QkJlWcIiIxVU1TPSXUpCsTP9B6iy22CLb5534ugkZQXKuTmuoiIsWS9HCklcCX2ce0aU/jy9294UNSqeLj6ltW0aFk2eeKa90qPq71KGlcE22qA5jZ22ls1qS13ElJ6/uT1nInJa3vT6nLraa6iEhMqjhFRGIqR8U5ogznLIa0ljspaX1/0lrupKT1/SlpuRPv4xQRSTs11UVEYkqs4jSzw8zsAzObb2aXJXXeuMysm5lNNbM5ZjbLzIZmt7czs8lmNi/72LbcZa0UaYit4hqf4lrPeZNoqptZM2Au0B9YDEwHBjnnZpf85DFl15zu7JybaWatgBnAkcCpwCrn3C3ZP6K2zrlh9bxUk5CW2Cqu8Siu9Usq49wHmO+c+9A5tx4YBwxM6NyxOOeWOudmZp9/DswBupAp7+jsYaPJBEdSElvFNTbFtR5JVZxdgEWR3xdnt1U0M+sB7EFmTeqOzrmlkAkW0KF8JasoqYut4poXxbUeSVWcudZ5rujL+WbWEpgAXOCcW9vQ8U1YqmKruOZNca1HUhXnYqBb5PeuwJKEzh2bmTUnE4Qxzrm/Zjcvy/an+H6V5eUqX4VJTWwV11gU13okVXFOB3qZ2fZm1gI4AXg6oXPHYpn5x0YCc5xzd0R2PQ2ckn1+CvBU0mWrUKmIreIam+Ja33mTGgBvZj8H7gKaAY84525M5MQxmdn/Av8A3gf8al5XkOk3GQ/8AFgIHOucW5XzRZqYNMRWcY1Pca3nvLpzSEQkHt05JCISkypOEZGYVHGKiMSkilNEJCZVnCIiManiFBGJSRWniEhMqjhFRGL6f2FY5IDaRuwVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "datagen = ImageDataGenerator(rotation_range=1.5)\n",
    "# fit parameters from data\n",
    "datagen.fit(Xtrain)\n",
    "# configure batch size and retrieve one batch of images\n",
    "for X_batch, y_batch in datagen.flow(Xtrain, ytrain, batch_size=9):\n",
    "    # create a grid of 3x3 images\n",
    "    for i in range(0, 9):\n",
    "        plt.subplot(330 + 1 + i)\n",
    "        plt.imshow(X_batch[i].reshape(28, 28), cmap=plt.get_cmap('gray'))\n",
    "    # show the plot\n",
    "    plt.show()\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(42000, 28, 28, 1)\n",
      "(42000, 28, 28, 1)\n",
      "(18000, 28, 28, 1)\n",
      "(84000, 28, 28, 1)\n"
     ]
    }
   ],
   "source": [
    "# Data Augmentation\n",
    "# Create copies of the original dataset\n",
    "Xtrain2 = np.array(Xtrain, copy=True)\n",
    "ytrain2 = np.array(ytrain, copy=True)\n",
    "\n",
    "# Create the image generator \n",
    "datagen1 = ImageDataGenerator(width_shift_range= 0.2, #Horizontal shift\n",
    "                             fill_mode = 'nearest',\n",
    "                             height_shift_range=0.2, # Vertical shift\n",
    "                             shear_range = 0.2,\n",
    "                             zoom_range = 0.2,\n",
    "                             rotation_range=25, # Random Rotations \n",
    "                             )\n",
    "\n",
    "# Fit the image generator to the training data\n",
    "datagen1.fit(Xtrain)\n",
    "\n",
    "# Concatenate original data with augmented data\n",
    "Xaug = np.concatenate((Xtrain, Xtrain2), axis = 0)\n",
    "yaug = np.concatenate((ytrain, ytrain2), axis = 0)\n",
    "\n",
    "print(Xtrain.shape)\n",
    "print(Xtrain2.shape)\n",
    "print(Xtest.shape)\n",
    "print(Xaug.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(42000, 10)\n",
      "(42000, 10)\n",
      "(18000, 10)\n",
      "(84000, 10)\n"
     ]
    }
   ],
   "source": [
    "print(ytrain.shape)\n",
    "print(ytrain2.shape)\n",
    "print(ytest.shape)\n",
    "print(yaug.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training a Second CNN with the Generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train with generator \n",
    "datagen_flow = datagen1.flow(Xaug, yaug, \n",
    "                            batch_size = 32)\n",
    "\n",
    "model2 = getModel()\n",
    "\n",
    "history_datagen = model2.fit_generator(datagen_flow, \n",
    "                                              steps_per_epoch = len(Xaug) // 32,\n",
    "                                              epochs = 20, \n",
    "                                              verbose = 0,\n",
    "                                              #callbacks = [es],\n",
    "                                              validation_data = (Xtest, ytest))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluating Scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Final non_augmented validation loss: 0.019426436877566934 \n",
      "final non_augmented validation accuracy: 0.9944999814033508\n",
      "Final augmented validation loss:  0.04878174902291762 \n",
      "final augmented validation accuracy: 0.9851666688919067\n"
     ]
    }
   ],
   "source": [
    "# Print Xtest scores\n",
    "print(\"Final non_augmented validation loss:\" , history_original.history['val_loss'][-1],  \"\\nfinal non_augmented validation accuracy:\", history_original.history['val_accuracy'][-1])\n",
    "print(\"Final augmented validation loss: \", history_datagen.history['val_loss'][-1], \"\\nfinal augmented validation accuracy:\", history_datagen.history['val_accuracy'][-1]) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABIEAAAIoCAYAAAD+9fvdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzde5hddX0v/vc3CUkgBMIlgFyDFuVWQIjaox4VEatUQSleEKi9CPZYtU9rfWqPPWptPb9TtWrrQa0eL61RKGqreEWsd61KqFVBRRERUlTCJUDut/X7Y+3d2ZnsmdmTzGQn8329nmc96772Z++ZZL77vb5rrdI0TQAAAACY2WYNuwAAAAAApp8QCAAAAKACQiAAAACACgiBAAAAACogBAIAAACogBAIAAAAoAJCIGCPVEr5k1LKp4ZdBwDAnkY7CuolBAJ2qVLK80op352CQ52S5NtTcBwAgD2CdhSws4RAwK728CT/PgXH2SMaL6WUOcOuAQCYMbSjgJ0iBIJxlFKeXEq5o5Ty+6WUm0opa0op/1JK2atnm9NLKdeWUlZ1tn11z7q9SymbSymXllK+UUpZXUr591LKsQO89qxSyj+WUn7W2e/mUspze9ZfVkr5+qh93lNKeVPP/BmllC+VUu4rpVxXSnlhKeXnPeu/XEp5Qynl6lLKA533eGrnLNONndf9x1JK6dnnEaWUz5ZSVnbe7//sWffQzvu9uJTy3c7n9aVSygGd9R9O8sdJnts59hc7yxeVUt5SSrml8zl+vJRyWM9xF5dS3l9KubOUcmsp5RlJTsgYjZdSyoGllA936ltTSvleKeWsUds8vZTylVLKXZ3hzT3rHlNKuaaU8stOPR/qLH9yKeWOUcd5VSnln3vmbyul/Hnns12T5OxSykNKKZ8spfyiU883Symn9exTSinP7/yMVpVSfl5KeXkp5eBSSlNKeeio34vlpZQX93vvAIB2lHaUdhSMRQgE4zs9yUFJ9klyapKHJnl8kicnSSnl4UmuTfLeznaPTfLCUspzOvufmmR2kkcneXqSg5M8kORFA7z2/kk+nfZMzX5J3pzkfaWUuZ31ZyS5ftQ+Z6RzdqiU8qtJ/jXJ3ydZnOQPkrwxyX901pckp3Xez8uTHJjkliQfTdsweERnuDDJ0s4+T+i833cleVCSM5O8pJTylJ7PK52aH5vk8M52z0uSpmkuSLIyya83TbNv0zRPKKXsk+QLSRZ2XudBSe5P8pbOay5I8sXOfscneVSSv0iyrlNvP4uSvC/JcZ3pTyT5f92VpZQ/SnJ5ktd0Xu9hSa7urPvNzvQ7kxyd5KiefSf6zA/ubP+UJL/XeU+fS/vze0OSJWl/Fjel/Xl2vSXJy9L+jA7sfA5fbZrmriQ/7rznrt9NMi/J28d47wCAdpR2lHYU9Nc0jcFgGGNIclWSK0Yt+2mSp3Wmr03yF6PW/2OSt3SmX5TktiRzeta/N8kbd6CWfZM0SQ7ozF+f5Hd61u+dZFOSEzvznxn9Okm+leR/d6Yf1jne8T3r35jkS6P2eSDJf+tM35Tkj0etf3f3M0jy+iT/Nmr9F5K8uDN9VJKtSfbrWf9naRtUs3qW/fckP+tMvyLJdaOO+f8l+eIkPruTk9zXmT46bcPncX22W5DkriQXj3Gcj/T5ef9nknM607+eZGOSIyao52lJvtOZ/rUka5I8dIxt/yHJWzvT+yf5ZZInDvvfhsFgMBgMe9KgHbXNMbWjdoPfSYNhWIOeQDC+05N8rDtTSlmU5Jgk3yulzE97BuefRu2zOO0ftO7+n2qaZnPP+tOSfG+iFy6lnF1K+XQpZUUp5b4kP0/y86Zp7u2cxTo5255NOTXJhiQ/LKUsTPKkJO8ZddiD0zmDlfbMyw+bpvnhqNp63++StH/QbyilnJi2J9SrO11tV5VSViV5TpL7et7vx7KtU3ve79IkP26a5v6e9c9M25C6p+eYn0hyT8/6K0Yd80EZ5zr2UspzOt2n7+h8dt9McmNn9dOSfL9pmi/32fXxaRt0Hxjj0Nucwep0tT48I9fmn5G2UfWfvTuVUl7U6br8y877+1BPPecn+UzTND8a4zW/npEzWK9O8pWmaT4/xrYAQLSjoh3VpR0FowiBYAyllP2TPDjtWZ+uM5Lc1TTNz9I2BGYnubdnnwelPfvypZ7tv9Wzfl6Sk5JcN8FrPyrJlUn+Lu2Zjf2T/HWS5Z1Njk9Skny/Z7dz0p4V2Zr20rTRtZ2ethtt94/+GX3qOL3nNbrb/Khpmgc67/eepmn2b5pmUc+wb9M0b+rZv/f9PiTtWZfeP+6juwEfnOTZo465f9M0D++sPzTJL3qOuTDtpXVjXcd+Qdozaa9Mcmzns7ui530dnOTufvt21t3bNE3T57j7pw0A/6Nn8TlJ7miaplvfGUm+MWq/P0lyaZIXpj2ztSjJ1wasJ0n+LcmppZRT03aN/pNxtgWA6mlHaUf10I6CUYRAMLbT0/4h671eemlG/uj8Z9pLvX67c5O5hyb55yQfbprmW52zTCdl28bAaemcZZrgtR+etuHx9SRbSim/nba7b/dYJe2Zlv2SpJTyuCR/lJFGwn929n9BKWV25w/fsrTdZW/ubHNGb219Ghqj3++NSeaV9kaKe3WO+yullP/e2f/YtNeNXz9q/5s6jZ8kOaTPe/1m2vsoHdI5zoGllPM719p3X/e3SikLSimHp22IHJixz2AtTftzWZ5kTinl5Ume3/M+rkvy6FLKr3Ve75BSypmdddcnOba0NzucVdobLf569yPqjLuf+UlJ/mrU57XNZ9r7GSS5IcnCUsrrk5w1qp7fKKUc3znuUaWU/9az/w1J1qf93XpL0zS3jvG+AYCWdpR2VJd2FIwiBIKxnZ7+N69bniSdsxwXpO1me3+Sa5J8MskLOtv+apLNGemu2t3/252zTOO5Mm0j4460Z6kOSNsY6b72d9I2Rr5TSvly2u63P0rnD2nTNJvS/sH+3SSrkrw2ycfTXhPedBoGD8/2Z6tuappm9ahl13eOeXfnvV6a9ozLXZ06D+j5vG5umua+Ufv3vsZ7kiwt7ZMdul2sX5r287uhlPJA5/XO7DmL9LK0jZWfp/2Mb0gbpP1gjM/u7UnmdGpcnpHu0N3P7tOdz+PK0j51YnnasC5N09yY9uf3xk5NP0jyuM66VUn+Msk1pZSvpT2bdH1GbmZ4YNozXKMbL3/dOf6qtDeYvDXt9fzdRs870zbI/rVTzxeSHNHdufO78q0ke3WOBQCMTztKOyqd19WOglFKn956wAxUSvlKkquapnnrsGthcKWUOWkbbK9qmuaqYdcDADXSjtozaUfB9vQEghmolHJYKeWsTlfjA0spf5v2mul3D7s2Btc50/gXSW7TcAGAXUM7ambQjoL+hEAwJKWUd5RSVo8xHL2Thz8syd+nfSzp99J2NX5S0zRrd7Zudo3OjRlXpb3R+CVDLgcAaqIdtYfTjoKxuRwMAAAAoAJ6AgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABeYM64UPPvjgZsmSJcN6eQBgml1//fV3NU2zeNh1sC1tMACY2cZrgw0tBFqyZEmWL18+rJcHAKZZKeVnw66B7WmDAcDMNl4bzOVgAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFCBCUOgUsp7Sil3llJuGGN9KaX8XSnl5lLKd0spp099mQAAddEGAwCm2iA9gd6X5CnjrH9qkuM6w2VJ3r7zZQEAVO990QYDAKbQhCFQ0zRfTnLPOJucl+Qfm9Y3kiwqpTxoqgoEAKiRNhgAMNWm4p5ARyS5vWd+RWcZAADTRxsMAJiUqQiBSp9lTd8NS7mslLK8lLJ85cqVU/DSAADV0gYDACZlzhQcY0WSo3rmj0xyR78Nm6Z5Z5J3JsnSpUv7NlIAABiINhiw29uyJXnggeT++9vhgQeSTZuSpvM/0SDjyWzbNXv2+MOsWRNvM95+TZNs3doO/aanalm/15jssn7LS2nfS/f9DDo96LazZydz5iR77bXteM6cdptdbcuWZMOGbYeNG7df1rt8y5aR36veYZBlg2xz2GHJ+efv2s8hmZoQ6OokLy6lXJnkUUnua5rm51NwXAAAxqYNxpTZujVZs6b9kn7ffWOP16xpvxht2dLu050ea5hom9Hrt24d+SI5lV9Ie6dHfzGbjqHfl/DxvqBPZl0pyd57J/vsMzLuDr3z460ba36vvdrjN83I70NvgDN6fpDptWuH/dvN7mbWrP4B0WTGs2dvG+KMFeh013UDnd3JYx6zm4ZApZQrkjwhycGllBVJXp1kryRpmuYdST6V5JwkNydZm+R3pqtYAIBaaIPtWZqm/bJ7773JPfdsO+5Or1rVbtvvzPjOfBnqHW/cOHGQ0298//3b9qIYy957t68zaE+K8bbpfpHr3WbWrJHAoxsK9U5v3rxteDR6/UTTW7a0Icd0D73BU3fo7XnR7Q3Rb914+82a1b6X9evb37e1a5Nf/jJZt25kvjts3Tr53+PZs5P589v9B/l92GuvZL/92mHhwnZ86KHJcceNzPeu607PndvuX8rg48ls2zQ7F04Osl+/n093emeXjf4d6vf7Meiyfsu7n9Ug/3Z25N9Z999qd9i0abDxINusWzey7dy5ybx5yYIFyYEHjsyPHnZ0+ezZ2/7+jf493Nll3X8Hu9qEIVDTNBdOsL5J8gdTVhEAsJ2maRtAgw4bNw6+7Ytf3H4hYfeiDTYcGzZsG9yMNz162aZNYx931qxk0aK24d/7pWa8fabS3nu3X8D3339kfOihI/Oj1/UbL1w4nMs4mLzu34y1a7cPiMabX7euHfbZZ+T3ol+I052eN2/Y7xSYLE0+gEkar7v26LMhk+nqPdEwet8tW9oGXr9usP2md2T9RF1nJzpLOMhZxGTHz/xNZl2/15vssn7bjO7+3++SgJ2Z37Jl+rsxX3qpEIjd1+rVyY03Jt/9bnL77duGJ6PDlJ2d37Ch/QI8nv33Tw44oB0OPDA54oiR6d7lo6fHC1C2bJm6M+Zz524f7PT2vKAO3V4Gc+e24SM7oWna/xx6U7L168fv5jZRN7l+DYrxXn/jxvY1u689erwjyzZtmtxNhCa7Lmm7is2dO/Z4vHWD7LtgQbLvvtsO3WVz507uc55OW7eOpK1r1oyM581LTjlll5ejyQdsZ+vWkZ4E3aHf/OhG6GSGQffZka66k+3SO9kwZ9BgY3cxZ07/bq/d6e540aLt1w8SDEz093Wi9Tt6I8jJrOv3epNdNtY23cCptwv3WMsmO9/tDt5t80x2GHS/ffbZ/r3BrrZ1a/LTn7ZhT+/wk59s+++v93e3exnU6OnR8/Pmtd8JJtpv7tyR8KZfmLP//tMTmHa/F+pVUYneGzB1b57zwAPbTo+3bt267e/Au6N3Ph5rm9Ff0HuHnVnWXZ5M7c2lRm+zcePIl+5ucNPb7Wky43Xrpr7x1/2jP97PpRv8rF+/c68/Z057jd/8+W2XwO70XnuNfw1iv2saJ3PtYm8X5t4vEBs2tOl+7xeMscabN+/c++4XDo0VGvVb3u1O1xvcTDTut2ysswuPeUzy1a/u+HvcQUIgmITeEwE7O3S7f0/m+ubueNBroUeHN4MEO8O4cVr32vjeoftEgUGf1NC7T/ca3kHbPaNvGjneMJntev9+jvf3cpBhrP277amxgp25c3XdB3Y/q1Yl3/vetmHP977XtpWT9v+t445LHv7w5PnPb0+UnnJKcswxu8+J3d3O+vXtNWkbN/b/A7mzvRFmou5Zr4m6yI7XZbbfeO3a8cOcBx4YrL45c0auveqODzig7X42XiiycePOByvdRuGeduZrIrNmbXtX7L33HplesCBZvHj7u273btMdz5s39uc30Wc/6M9m7tyR0KY3vJnssj25q+/oIKl3vGFD+0dj9ep26J3uHUYv//nPt1822S8/pYzcUX3Bgm3Hhx7af3m/8SGHTM/nNoE9+DeC3VH3xMbatSO9OMbq3bGzy7vLuv9XjrdsR9b33nCvd9iRv4Wjn76wzz5t+D5eT4axxoNum2x/0mXBgv4nZ0afqBlkXe8Z1N6h37KJBiEFwMy0eXPy4x9v37vntttGtjnwwOTUU5MXvGAk7DnxxIp7qG3c2IY5d9/djnunx1u2I49g6tcTYazQqPdu0NMZHg3y2K0dfVxXd7r3i+RUmjWrDQe6N2BauLAdDjooWbJk2zCnd9xvWfeGO8MM6rrXJI8+azjW2cTJLEt2rgfTRNt3u7mODnG6jz9jz9B7XeOCBdPzGt2z/P1CpO4XqNHhzfz5e/TvkRCoYt3AZvXq9oTE6N/5QZf1Lh/mIyC7///360XSb3q89d1LY0YHNzsyzJ8v5ABg5lu5cvuw58Yb27Z10v59Pf745LGPHQl7TjklOfzwIbSlt2xJfvGL9iZDt93Wjm+/Pbnzznb9dFxe0113//3jhzqrV49d95w5baBw4IHt+Jhj2u5S3WUHHtg2Yvr1LOh3jfVY112Pt3xHHjk1WWM9cqvfNbODDKP3614j2K/7bL/xoNvuyT0u+ill5Izd3nsPuxqYHqWM9J466KBhV7NLzLD/qeq1aVPbdli5Mrnrrm3H/Zbdd9/kApu5c/tfKnnwwf2X7713/x4h3ZBlqpbr0QwAw9c0bcBzzz3t/GGHtQHPS14yEvYcf/wuuudN07SNot5wpzt0l91xx/b3mth337bwZPKXb0y2m/Ds2SOhzUEHtZf3nHLKtsv6jffdV4MHgJ0iBNoNNU3bs2Z0cDPeeNWqsY93wAHt5a0HH5wce2zyiEe0y/qFNwsX9l/uiRIAwFhKSd7+9jar+NVfnebbHNx///bhTm/As2LF9jfhnDs3OfLI5Kijksc/vh2PHrrPb98R3cuNBrkPSPexXcIcAIZACDRE3adgfP/7bXfp7viHPxy7l87cuSOBzuLFbS/g3vne6YMPbhtjM61nKgCw+3n2s6fpwFu3Jtdem7ztbcmXvtR2Z+41a1byoAclRx/dXhp17rkjwc7RR7fjxYun99rs7pMAZs+evtcAgCkgHtgFtmwZO+zpPVF15JHtjRAvu6ztFTw60Fm8WC9gAKAS99yTvPe9bRejn/yk7V504YXJgx+8bQ+eww93xgsABuQv5hTasiW55Zb+Yc/69SPbHXVUG/aceWY7Pumk5IQTkv33H17tAAC7heXL214/V1zRNqAe+9jkL/8yOf/8XXRTIQCYuYRAO2DLlvaEVL+wp/sEjKTtgXziiclZZ20b9uy33/BqBwDY7axbl1x1VRv+fOtb7SN4f/u3k//xP9obJgMAU0IINEkf+Ujy/Oe3j1bvOuaYNuQ5++xtw56FC4dXJwDAbu+WW9rLvd7znvbyrxNOSN761uSSS3SRBoBpIASahOuuSy6+uH3qxYte1IY9xx8v7AEAGNiWLcmnP932+vnMZ9obNj/zmW3j6glPcPNDAJhGQqABrViRnHdecthhySc/2d6kGQCAAa1c2fb4ecc7kltvbZ/o9epXJy94QftEDABg2gmBBrBmTfu00dWr2yeUCoAAAAbQNMk3v5lcfnl7z5+NG9snY7zhDe3Ztb32GnaFAFAVIdAEtm5tLwH7zneST3yivQQMAIBxrFnTPt3rbW9Lvv3t9qkYL3xh8vu/395AEQAYCiHQBP7n/0w++tHkLW9JnvrUYVcDALAb+9GP2hs9v/e9yX33tTdSfMc7kosuSvbdd9jVAUD1hEDjeN/7kr/+6/bE1UtfOuxqAAB2Uxs3Jk9/evLZz7aXeF1wQfIHf5A8+tFu9AwAuxEh0Bi+8pXkssuSs85qn1Sq/QIAMIa5c5Mjj0xe97rk934vOfTQYVcEAPQhBOrjllvaJ5Uee2zyoQ+5ZyEAwITe/e5hVwAATGDWsAvY3dx3X/K0p7U3hP7EJ5IDDhh2RQAAAAA7T0+gHps3J895TvLjH7eXtB933LArAgAAAJgaQqAef/zHyTXXJO98Z3LmmcOuBgAAAGDquBys4+1vb28A/Ud/lFx66bCrAQAAAJhaQqAk116bvOQlyW/8RvKGNwy7GgAAAICpV30I9MMfJs96VnLCCckHP5jMnj3sigAAAACmXtUh0N13t08Cmzs3+fjHk/32G3ZFAAAAANOj2htDb9yY/OZvJrffnnzhC8mSJcOuCAAAAGD6VBkCNU3yohclX/pSsmxZ8uhHD7siAAAAgOlV5eVgb3pT8u53J698ZXLRRcOuBgAAAGD6VRcCffzjyctfnlxwQfLa1w67GgAAAIBdo6oQ6LvfTZ73vOT005N/+IdkVlXvHgAAAKhZNTHIL3+ZPP3p7RPAPvaxZJ99hl0RAAAAwK5TxY2h169PnvGMZOXK5CtfSY44YtgVAQAAAOxaMz4Eaprkd383+cY3ko98JDnjjGFXBAAAALDrzfgQ6K/+KrniiuR1r0vOP3/Y1QBQrS1bko0bk02bth1PtGzr1mT27PZGdmONd3Rdd3zggUkpw/6EAACYZjM6BLrqquRVr0ouuST5sz8bdjUAu7GmSTZsSNas2XbYuLENL7ZubYfu9Ojxjq5rmvb1uwFEv/HOrGua9rU2b56+YXSAM1a4s3Xr9P38dtaaNW6WBwBQgRkbAl13XfL85yePfnTyrnc5wQnsoZpm21Bh48Y2rOlOjw5tBhlWr+6/fHcOKabLXnslc+b0H2bPHntdd5g3rz3G3Lnt0J0edNkg+8yaNT0BXO82c+cO+ycBAMAuMCNDoBUrkvPOSw47LPmXf2nb6ADb2bp123BldMAy1vRkthtrWb+h33abNu34+yslWbCg/7B48cj0vvuOvd28eYNfUrQj67q9dZL+451dl4wd4Myq5gGZAACQZAaGQGvWJOee257o/uxnk0MOGXZFwA5rmvbxfv16raxdO35vl0HWb9gw9TXPmdP2quj2EJk3b6R3R3foLtt33+3X9dtuvGGffbYPbrqhzvz5ukECAAD/ZcaFQC98YfKd7yQf/3hy8snDrgZmiO4lSevXt8O6dWOPx1s36Lg7vXbttj06BrH33v17tDzoQdsHJnvvPRK0jA5ceufHmu43r3cJAACwm5pxIdCf/Ely5pnJOecMuxLYhZqm7d2ycmU73Hlncu+9YwctO7JsZ+4XM3t2G7jMn99/fMAByeGHj8x3140Obfr1ehkd6ghhAAAA+ppxIdBpp7UD7NGapr2msTfU6U6PNb9+/cTHnTt3+xCmd3rx4vHXjzXdO+63bM6M+68GAABgj+ObGUy3DRuS++5rh/vvH5m+557xQ52x7lezzz5tWLN4cXLooe11j935Qw4ZmT7wwHbbbhDTvcEvAAAAVRICwVi2bm0vseqGNqNDnLGmR89v3Dj+63Sf1LR4cftIu1NOGZkfHex0n+gEAAAAkyQEok6bNyd33JHcdlty++3bj2+/ve2pM9FNiUtJ9tsv2X//dthvvzbIedjDRuZ713Wn99+/vQ/O4sVtbx0AAACYZkIgZp6mSe6+e+yA57bb2gBo9I2OFy1Kjj46Oeqo5Nd+LTn44P4BTu/0ggUusQIAAGCPIARiz7NuXRvkdId+Qc+6ddvuM29eG+4cdVRy1lnt+OijR0Kfo45KFi4czvsBAACAXUAIxO7ngQeSn/2sHW69dfvxnXduu30pyYMe1AY6p56aPP3pIyFPd7x4cbsdAAAAVEoIxK63alX/cKc7vueebbefNy855ph2OO+8keljjmlDnsMPbx99DgAAAIxJCMTUu//+5Ec/Gjvouf/+bbffZ59kyZI21HnUo9pxd37JkvbpWO67AwAAADtFCMSOW7s2+cEPkhtuaIcbb2zHt9++7Xb77deGOUuWJE94wvYhz0EHuVQLAAAAppkQiIlt2JDcdNO2Qc+NNya33DLyCPV585Ljj08e97jkpJOSE04YCX4WLRpm9QAAAECEQPTavDn58Y+3DXpuuKFdtmVLu82cOclDH5qcfnryW7+VnHxyG/o85CHtOgAAAGC35Ft7jbZuTX76022DnhtuaHv7bNzYblNK8iu/0gY8F1zQjk8+uQ2A3IQZAAAA9jhCoBo0TfKTnySf+1w7fP7zyb33jqw/5pg24HnqU0d69hx/fHvDZgAAAGBGEALNVCtXtmHP5z6XXHtt+1SupH2k+jOfmTz60W3gc+KJycKFw60VAAAAmHZCoJli7drkq19tA5/PfS75j/9ol++/f/LEJyZ/+qfJk57UXuLlSVwAAABQHSHQnmrLluT660cu8fra19r7+cyd2/byed3r2tDnjDOS2bOHXS0AAAAwZEKgPUXTJDffvO19fVatateddlry0pe2oc9jH5ssWDDcWgEAAIDdjhBod3bnnW3Y073E67bb2uVHH5385m+2oc8Tn5gccshw6wQAAAB2e0Kg3c0ddyRvfWvy6U8n3/lOu2zRojbsecUrkrPPTh7yEPf1AQAAACZFCLS7+MUvkv/zf5K///tk06bkcY9r7+tz9tnJ6ae7rw8AAACwU4RAw/bLXyavf33ytre14c9v/Vby53+ePPjBw64MAAAAmEGEQMOycmUb/lx+ebJhQ3Lxxcn/+l/tI9wBAAAAppgQaFe7667kjW9M/u//TdatS573vDb8eehDh10ZAAAAMIMJgXaVe+5J/uZvkr/7u2TNmuS5z01e9ark+OOHXRkAAABQASHQdLv33uRNb0r+9m+T1auTZz0refWrkxNPHHZlAAAAQEWEQNNl1Qh6ytEAACAASURBVKrkLW9J3vzm5P77kwsuaMOfk08edmUAAABAhYRAU+2++9peP29+cxsEPfOZyWtek5xyyrArAwAAAComBJoqDzzQ3u/nb/6mvQTsvPPa8Oe004ZdGQAAAIAQaKetXp289a3tE7/uuSd52tPa8OeMM4ZdGQAAAMB/EQLtqDVrkssvT97whvax7+ec04Y/j3jEsCsDAAAA2M6sYRewx9mypX3a14MfnPzpn7Y9fr7xjeSTnxQAAQAAALstIdBkvetdycte1t7o+etfTz7zmeRRjxp2VQAAAADjcjnYZP3DP7QB0LXXDrsSAAAAgIHpCTQZN9/cXvp18cXDrgQAAABgUoRAk/GBDySlJBdeOOxKAAAAACZFCDSopkne//7kzDOTI48cdjUAAAAAkyIEGtQ3v5n85CfJJZcMuxIAAACASRMCDWrZsmT+/OT884ddCQAAAMCkDRQClVKeUkq5qZRycynlFX3WH11K+UIp5dullO+WUs6Z+lKHaNOm5Mork/POS/bbb9jVAACVqL4NBgBMqQlDoFLK7CSXJ3lqkhOTXFhKOXHUZn+e5KqmaR6e5LlJ3jbVhQ7VZz6T3H23S8EAgF1GGwwAmGqD9AR6ZJKbm6a5pWmajUmuTHLeqG2aJN0uMvsnuWPqStwNLFuWHHxw8uQnD7sSAKAe2mAAwJSaM8A2RyS5vWd+RZJHjdrmNUk+W0p5SZIFSZ40JdXtDu67L7n66uQFL0j22mvY1QAA9ai7DQYATLlBegKVPsuaUfMXJnlf0zRHJjknyftLKdsdu5RyWSlleSll+cqVKydf7TB85CPJ+vXJxRcPuxIAoC51t8EAgCk3SAi0IslRPfNHZvuuxr+X5KokaZrm35LMT3Lw6AM1TfPOpmmWNk2zdPHixTtW8a62bFly3HHJIx857EoAgLrU3QYDAKbcICHQdUmOK6UcW0qZm/amg1eP2ua2JGclSSnlhLQNkD3/NNPttydf/GLbC6j0OxkHADBt6m2DAQDTYsIQqGmazUlenOSaJD9I+wSKG0spry2lnNvZ7GVJLi2lfCfJFUl+u2ma0d2V9zxXXJE0TXLRRcOuBACoTNVtMABgWpRhtROWLl3aLF++fCivPZCmSU45JVm4MPn614ddDQDscUop1zdNs3TYdbCt3b4NBgDslPHaYINcDlan7343ueGG5JJLhl0JAAAAwE4TAo1l2bJkzpzk2c8ediUAAAAAO00I1M+WLckHP5icc05y0EHDrgYAAABgpwmB+vnCF5I77nApGAAAADBjCIH6WbYs2W+/5GlPG3YlAAAAAFNCCDTa2rXJRz6SPOtZyfz5w64GAAAAYEoIgUb72MeS1auTiy8ediUAAAAAU0YINNqyZclRRyWPe9ywKwEAAACYMkKgXnfemVxzTXLRRcksHw0AAAAwc0g6el15Zft4eJeCAQAAADOMEKjXsmXJaaclJ5007EoAAAAAppQQqOumm5LrrksuuWTYlQAAAABMOSFQ1wc+0N4H6LnPHXYlAAAAAFNOCJQkTdNeCnbWWcnhhw+7GgAAAIApJwRKkq9/PfnpT90QGgAAAJixhEBJ2wton32S888fdiUAAAAA00IItGFD8k//lDzjGcm++w67GgAAAIBpIQT69KeTe+91KRgAAAAwowmBli1LDjkkOfvsYVcCAAAAMG3qDoFWrUo+/vHkwguTOXOGXQ0AAADAtKk7BPrQh5KNG10KBgAAAMx4dYdAy5YlD3tYcsYZw64EAAAAYFrVGwL97GfJl7+cXHJJUsqwqwEAAACYVvWGQB/4QDt+3vOGWwcAAADALlBnCNQ0yfvfnzz2scmxxw67GgAAAIBpV2cI9O1vJz/8oRtCAwAAANWoMwRatiyZOzd59rOHXQkAAADALlFfCLR5c/LBDya/8RvJAQcMuxoAAACAXaK+EOhf/zX55S9dCgYAAABUpb4QaNmyZNGiticQAAAAQCXqCoFWr07++Z/bewHNmzfsagAAAAB2mbpCoI9+NFm71qVgAAAAQHXqCoGWLUuOOSZ5zGOGXQkAAADALlVPCPSLXyTXXtv2AppVz9sGAAAASGoKga64Itm6NbnoomFXAgAAALDL1RMCLVuWnHFGcsIJw64EAAAAYJerIwT6/veTf/93N4QGAAAAqlVHCLRsWTJ7dnLhhcOuBAAAAGAoZn4ItHVr8oEPJGefnRx66LCrAQAAABiKmR8CffWryW23uRQMAAAAqNrMD4GWLUsWLEie8YxhVwIAAAAwNDM7BFq/PrnqquT889sgCAAAAKBSMzsE+uQnk/vucykYAAAAUL2ZHQItW5YcdljyxCcOuxIAAACAoZq5IdDdd7c9gZ73vGTOnGFXAwAAADBUMzcE+tCHkk2bXAoGAAAAkJkcAi1blpx4YnLaacOuBAAAAGDoZmYIdMstyde+1vYCKmXY1QAAAAAM3cwMgT7wgXZ80UXDrQMAAABgNzHzQqCmaS8Fe/zjk6OPHnY1AAAAALuFmRcCLV+e/OhHbggNAAAA0GPmhUDvf38yb15ywQXDrgQAAABgtzFn2AVMuSc9KTniiGTRomFXAgAAALDbmHkh0LnntgMAAAAA/2XmXQ4GAAAAwHaEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVGCgEKiU8pRSyk2llJtLKa8YY5tnl1K+X0q5sZTywaktEwCgPtpgAMBUmjPRBqWU2UkuT3J2khVJriulXN00zfd7tjkuyZ8leUzTNPeWUg6ZroIBAGqgDQYATLVBegI9MsnNTdPc0jTNxiRXJjlv1DaXJrm8aZp7k6RpmjuntkwAgOpogwEAU2qQEOiIJLf3zK/oLOv10CQPLaV8rZTyjVLKU/odqJRyWSlleSll+cqVK3esYgCAOmiDAQBTapAQqPRZ1oyan5PkuCRPSHJhkv9XSlm03U5N886maZY2TbN08eLFk60VAKAm2mAAwJQaJARakeSonvkjk9zRZ5uPNU2zqWmanya5KW2DBACAHaMNBgBMqUFCoOuSHFdKObaUMjfJc5NcPWqbjyY5M0lKKQen7Zp8y1QWCgBQGW0wAGBKTRgCNU2zOcmLk1yT5AdJrmqa5sZSymtLKed2Nrsmyd2llO8n+UKSlzdNc/d0FQ0AMNNpgwEAU600zehLy3eNpUuXNsuXLx/KawMA06+Ucn3TNEuHXQfb0gYDgJltvDbYIJeDAQAAALCHEwIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRgoBCqlPKWUclMp5eZSyivG2e6CUkpTSlk6dSUCANRJGwwAmEoThkCllNlJLk/y1CQnJrmwlHJin+0WJnlpkm9OdZEAALXRBgMAptogPYEemeTmpmluaZpmY5Irk5zXZ7u/TPL6JOunsD4AgFppgwEAU2qQEOiIJLf3zK/oLPsvpZSHJzmqaZpPjHegUsplpZTlpZTlK1eunHSxAAAV0QYDAKbUICFQ6bOs+a+VpcxK8uYkL5voQE3TvLNpmqVN0yxdvHjx4FUCANRHGwwAmFKDhEArkhzVM39kkjt65hcmOTnJF0sptyb5tSRXuzEhAMBO0QYDAKbUICHQdUmOK6UcW0qZm+S5Sa7urmya5r6maQ5ummZJ0zRLknwjyblN0yyflooBAOqgDQYATKkJQ6CmaTYneXGSa5L8IMlVTdPcWEp5bSnl3OkuEACgRtpgAMBUmzPIRk3TfCrJp0Yte9UY2z5h58sCAEAbDACYSoNcDgYAAADAHk4IBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRACAQAAAFRACAQAAABQASEQAAAAQAWEQAAAAAAVEAIBAAAAVEAIBAAAAFABIRAAAABABYRAAAAAABUQAgEAAABUQAgEAAAAUAEhEAAAAEAFhEAAAAAAFRgoBCqlPKWUclMp5eZSyiv6rP/jUsr3SynfLaX8aynlmKkvFQCgLtpgAMBUmjAEKqXMTnJ5kqcmOTHJhaWUE0dt9u0kS5umOSXJh5O8fqoLBQCoiTYYADDVBukJ9MgkNzdNc0vTNBuTXJnkvN4Nmqb5QtM0azuz30hy5NSWCQBQHW0wAGBKDRICHZHk9p75FZ1lY/m9JJ/emaIAANAGAwCm1pwBtil9ljV9Nyzl4iRLkzx+jPWXJbksSY4++ugBSwQAqJI2GAAwpQbpCbQiyVE980cmuWP0RqWUJyV5ZZJzm6bZ0O9ATdO8s2mapU3TLF28ePGO1AsAUAttMABgSg0SAl2X5LhSyrGllLlJnpvk6t4NSikPT/L3aRsfd059mQAA1dEGAwCm1IQhUNM0m5O8OMk1SX6Q5KqmaW4spby2lHJuZ7M3JNk3yYdKKf9RSrl6jMMBADAAbTAAYKoNck+gNE3zqSSfGrXsVT3TT5riugAAqqcNBgBMpUEuBwMAAABgDycEAgAAAKiAEAgAAACgAkIgAAAAgAoIgQAAAAAqIAQCAAAAqIAQCAAAAKACQiAAAACACgiBAAAAACogBAIAAACogBAIAAAAoAJCIAAAAIAKCIEAAAAAKiAEAgAAAKiAEAgAAACgAkIgAAAAgAoIgQAAAAAqIAQCAAAAqIAQCAAAAKACQiAAAACACgiBAAAAACogBAIAAACogBAIAAAAoAJCIAAAAIAKCIEAAAAAKiAEAgAAAKiAEAgAAACgAkIgAAAAgAoIgQAAAAAqIAQCAAAAqIAQCAAAAKACQiAAAACACgiBAAAAACogBAIAAACogBAIAAAAoAJCIAAAAIAKCIEAAAAAKiAEAgAAAKiAEAgAAACgAkIgAAAAgAoIgQAAAAAqIAQCAAAAqIAQCAAAAKACQiAAAACACgiBAAAAACogBAIAAACogBAIAAAAoAJzhl0AAEzWpk2bsmLFiqxfv37YpZBk/vz5OfLII7PXXnsNuxQAYBppg+1edqQNJgQCYI+zYsWKLFy4MEuWLEkpZdjlVK1pmtx9991ZsWJFjj322GGXAwBMI22w3ceOtsFcDgbAHmf9+vU56KCDND52A6WUHHTQQc4IAkAFtMF2HzvaBhMCAbBH0vjYffhZAEA9/N3ffezIz0IIBAA7YMWKFTnvvPNy3HHH5SEPeUj+8A//MBs3buy77R133JELLrhgwmOec845WbVq1Q7V85rXvCZvfOMbJ9xu3333HXf9qlWr8ra3vW2HagAAmG7aYDtHCAQAk9Q0Tc4///w84xnPyI9//OP86Ec/yurVq/PKV75yu203b96cww8/PB/+8IcnPO6nPvWpLFq0aDpKHpgQCADYXWmD7TwhEABM0uc///nMnz8/v/M7v5MkmT17dt785jfnPe95T9auXZv3ve99edaznpWnP/3pefKTn5xbb701J598cpJk7dq1efazn51TTjklz3nOc/KoRz0qy5cvT5IsWbIkd911V2699daccMIJufTSS3PSSSflyU9+ctatW5ckede73pVHPOIROfXU/7+9O4+Sqj7TOP59aZYGBBQVwcEFFRQboUO3oAJBgriFGANqREHRDC6IihoTEiUzLjGYGNFENGo0RsUVVIxjktFE54A6x8AEkda4oBg1RloSEQ4mdNPv/PG73ZaV6rXureqqfj7n9Olabr/vXarqPn23GsHUqVPZunVrk+P69ttvc+ihh3LwwQczf/78hse3bNnCxIkTGTlyJAcddBDLli0DYN68eaxbt47y8nIuvfTSRocTERERyTVlsOzp28FERKSgzZ0Lq1fHW7O8HG64ofHnq6qqqKio+NxjvXv3Zs899+TNN98E4IUXXmDNmjX07duX9evXNwx38803s9NOO7FmzRrWrl1LeXl5xh5vvPEG999/P7fffjsnnXQSS5cuZfr06UyZMoVZs2YBcPnll3PHHXdw/vnnNzquF154Ieeeey6nnXYaixYtani8tLSURx99lN69e/PRRx9xyCGHcNxxx7FgwQLWrl3L6mim1tbWZhxO1wMQERHp2JTBCjOD6UggERGRVnL3jCvg1McnTZpE3759/2WYFStWcPLJJwMwbNgwhg8fnrHHoEGDGsJJRUVFQ4hZu3Yt48aN46CDDmLx4sVUVVU1Oa7PPfcc06ZNA2DGjBmfG9fvfve7DB8+nCOOOIL333+fDz/8MOM0tWQ4ERERkaQpg2WfwXQkkIiIFLSm9hYlpaysjKVLl37usU8++YR3332Xfffdl1WrVtGzZ8+Mf+vuLerRrVu3htslJSUNhyLPnDmTxx57jBEjRnDXXXfx7LPPNlsrU1havHgx1dXVrFq1ii5durD33ntn/IrRlg4nIiIiHYsy2LPN1mqPGUxHAomIiLTSxIkT2bp1K3fffTcA27dv55JLLmHmzJn06NGjyb8dO3YsDz30EACvvPIKL7/8cqt6b968mQEDBlBTU8PixYubHX7MmDE88MADAJ8bftOmTfTr148uXbrwzDPP8M477wDQq1cvNm/e3OxwIiIiIrmmDJY9bQQSERFpJTPj0Ucf5eGHH2bw4MEMGTKE0tJSrrnmmmb/dvbs2VRXVzN8+HCuvfZahg8fTp8+fVrc+6qrrmL06NFMmjSJAw44oNnhb7zxRhYtWsTBBx/Mpk2bGh4/9dRTWblyJZWVlSxevLih1s4778yYMWMYNmwYl156aaPDiYiIiOSaMlj2rKWHRMWtsrLS66/ELSIi0hqvvvoqQ4cOzfdotMn27dupqamhtLSUdevWMXHiRF5//XW6du2a71HLSqZlYmar3L0yT6MkjVAGExGRtlIGa39am8F0TSAREZEc2rp1KxMmTKCmpgZ355Zbbin48CEiIiLS3imDBdoIJCIikkO9evVCR2GIiIiI5JYyWKBrAomIiIiIiIiIdADaCCQiIiIiIiIi0gFoI5CIiIiIiIiISAegjUAiIiIiIiIiIh2ANgKJiIi0QUlJCeXl5ZSVlTFixAiuv/566urqmvyb9evXc9999+VoDEVERESKjzJYdrQRSEREpA26d+/O6tWrqaqq4qmnnuLJJ5/kiiuuaPJvFEBEREREsqMMlh1tBBIREclSv379uO2227jppptwd9avX8+4ceMYOXIkI0eO5Pnnnwdg3rx5LF++nPLychYuXNjocHV1dcyePZuysjImT57Msccey5IlSwBYtWoV48ePp6KigqOOOooPPvgAgMMPP5xvf/vbjBo1iiFDhrB8+fL8zAwRERGRHFEGa73O+R4BERGRrMydC6tXx1uzvBxuuKFVf7LPPvtQV1fHhg0b6NevH0899RSlpaW88cYbTJs2jZUrV7JgwQKuu+46nnjiCQC2bt2acbhHHnmE9evX8/LLL7NhwwaGDh3KmWeeSU1NDeeffz7Lli1j11135cEHH+Syyy7jzjvvBKC2tpYXX3yxYY/Y008/He98EREREamnDFaQGUwbgURERGLi7gDU1NQwZ84cVq9eTUlJCa+//nrG4RsbbsWKFZx44ol06tSJ/v37M2HCBABee+011q5dy6RJkwDYvn07AwYMaKg3ZcoUACoqKli/fn1SkykiIiLSriiDtZw2AomISGFr5d6ipLz11luUlJTQr18/rrjiCnbbbTdeeukl6urqKC0tzfg3CxcuzDhcfZBJ5+6UlZXxwgsvZHy+W7duQLhgYm1tbQxTJSIiItIIZbAGhZTBdE0gERGRLFVXV3POOecwZ84czIxNmzYxYMAAOnXqxD333MP27dsB6NWrF5s3b274u8aGGzt2LEuXLqWuro4PP/yQZ599FoD999+f6urqhgBSU1NDVVVVbidWREREpJ1QBms9HQkkIiLSBp9++inl5eXU1NTQuXNnZsyYwcUXXwzA7NmzmTp1Kg8//DATJkygZ8+eAAwfPpzOnTszYsQIZs6c2ehwU6dO5Xe/+x3Dhg1jyJAhjB49mj59+tC1a1eWLFnCBRdcwKZNm6itrWXu3LmUlZXlbT6IiIiI5JIyWHasscOdklZZWekrV67MS28RESlsr776KkOHDs33aCRqy5Yt7LDDDmzcuJFRo0bx3HPP0b9//3yPVqMyLRMzW+XulXkaJWmEMpiIiLSVMlj709oMpiOBRERE2qHJkyfz8ccfs23bNubPn9+uw4eIiIhIsSj2DKaNQCIiIu1Q/TnoIiIiIpI7xZ7BdGFoEREREREREZEOQBuBRESkIOXrmnbyr7QsREREOg6t99uPtiwLbQQSEZGCU1paysaNGxVC2gF3Z+PGjZSWluZ7VERERCRhymDtR1szmK4JJCIiBWfgwIG89957VFdX53tUhBAIBw4cmO/REBERkYQpg7UvbclgLdoIZGZHAzcCJcDP3X1B2vPdgLuBCmAj8HV3X9+qMREREWmhLl26MGjQoHyPhkjilMFERKQ9UQYrfM2eDmZmJcAi4BjgQGCamR2YNtg3gL+7+37AQuDauEdUREREpCNRBhMREZG4teSaQKOAN939LXffBjwAfDVtmK8Cv4xuLwEmmpnFN5oiIiIiHY4ymIiIiMSqJRuB/g14N+X+e9FjGYdx91pgE7BzHCMoIiIi0kEpg4mIiEisWnJNoEx7k9IvBd6SYTCzs4CzortbzOy1FvRvi12AjxKqnYv6ueih+vnvUej1c9FD9fPfQ/Xz36OQ6++VUN2OQhms+Ornoofq579HodfPRQ/Vz38P1c9/j7xksJZsBHoP2CPl/kDgL40M856ZdQb6AH9LL+TutwG3taBnVsxspbtXFmr9XPRQ/fz3KPT6ueih+vnvofr571Ho9SUrymBFVj8XPVQ//z0KvX4ueqh+/nuofv575CuDteR0sD8Ag81skJl1BU4GHk8b5nHg9Oj2CcDv3f1f9kKJiIiISIspg4mIiEismj0SyN1rzWwO8FvC15Pe6e5VZnYlsNLdHwfuAO4xszcJe59OTnKkRURERIqdMpiIiIjErSWng+HuTwJPpj32vZTb/wBOjHfUspL04c6JH06dgx6qn/8ehV4/Fz1UP/89VD//PQq9vmRBGazo6ueih+rnv0eh189FD9XPfw/Vz3+PvGQw0xHDIiIiIiIiIiLFryXXBBIRERERERERkQKnjUCSCDPL9JW17Z6Z9cxBj/6FOn9ERESk/SrkfJF0BlP+EhEJimYjkJntb2aHmlkXMytJqEcidaPa+5lZpZl1S7BHmZmNN7OdE6o/1sxmALi7x72iNbOvmNmFcdZMq/9V4Foz65dgj6OAR/n8V/7GWf8QM5sR/e6aQP3B0eu0JMn3g0ickg79+qdCOrJc5K+oT8FmsELPX1GPgs5gSeevqIcymEgaZbD2qSg2ApnZFGAZcDXhWzLOM7PeMdYfAuDu25P40DWzycAjwI+Au+r7xdzjGOB+4CLgbjPrH2PtTma2A3Ar8B0zOwcagkgsrzEzOxK49I/fOwAADM9JREFUCngljnoZ6o8HrgWWufuGhHocGfUYAFySQP3jCBcXOwL4JrBXzPWPB5YA3wGuB87OxZFTUe+C/oDXCjAzM+uecP3+ED6LEqo/OMn6GfrpdSTtStL5K+pR0Bms0PNX1KegM1jS+SvqUZQZrBjWC1p3ZqYM1up+RfU6KviNQGbWBfg68A13n0gII3sA34ojiEThYLWZ3QfxhxAzOwy4Djjd3ScAfwfmxVU/6nE4cCPw7+5+PLANGBZXfXevc/ctwC8JIfAwM7uo/rls60fz6B7gLHd/ysz6mNleZtYj29opKoCfR/V3N7NJZjbazPrEUdzMjgBuBk4FBgNDzeyLcdSO6u8MnAec4u6nA58A5WbWz8xKY6p/NjDN3acCLwFnABeZWa9s62foNzraa3owxL9nM+5/UjLUHxntmR0F8a+gor3uR5vZpCTqRz2Oqd+znIRor+ycOF6fjdQ/BviJme2XUP1JwPNmdmYS9aMeXzKzWWY2CxJ5HY0yszFmVllfv1DDrORe0vkr6lHQGazQ8xcUfgZLOn9FPYomgyWdv6IeymBN1080f0U9lMGa71HUGazgNwJFehM+2CEc6vkE0BU4JZuZGW1hnwPMBbaZ2b2QyN6oBe7+x+j2fwB9Ld5Dkj8Eznb3F6OtsqMJb/xbzeyEGF9wtYQA+EtglJldb2Y/sCCb19pGoAYYEK0IHwNuIeyxi2v8a1NuLwHOJCz7RWa2Uwz1S4DT3L0K6Am8BpRBbFt+a4HuwAHRyvVw4DTgBuByy35vUS2wA1C/Vf9O4B1gV2BylrU/J1px3EsIbJeZ2R1Rz1g+HC3suV4eBZ3YPwOjf1ruAM4CvmlmZ8dc/1jgZ8CXgLkW9j7WPxfLezkKBecCt1k4RD9W0TL+IfCH6OutU5+LYxmPJuwZ/5m7v5n2XNbL3MyOJhw18Gui90QCIfkY4CdAH+BUM5uW8lwc8+jLwO3Al4ELzOxW0IYgabVE8hcUTQYr9PwFhZ/Bks5fUCQZLOn8FfVQBmu6fqL5K+qhDNZ8j+LPYO5e8D/AJOBxYFx0vwQ4hfBBZlnW3p3wwbsLYcV0b8zjXgL0Trk9EPgjsGv02M4x97sMuDy6fQbwYH2vGGrvC8yLbl8CbAUWxVR7BPAW8B4wi7AB80zCIdZ9Y6g/jBAMHgDOiB7bh/BBf1SM879T9Pto4K/AQTHWPgFYBfwvMD967EvAXcCIGOqfQ9gbOAP4fvT+Ohu4M8ZpKImWwYzofm9gBbAkZZg2v6eBvaN6T0V9KrP9jEir/wVgTf38Bk4EFsZYfySwEjg0un81cBzQL475k9ZrVjSP3ibsJW94/WZZ98Co5lnR/Z2B/VPfCzF8bk8Hvh/d3p2wkj0t5fk2Twch3P+RsOd61+h9PCmuZRz16An8FvhydH8OMA2ojGMeAT0I4WlidH9PYEOc72X9dIwfEsxfUb2iyWAUaP6KahZ8BiPB/BXVLegMRsL5K/r7vVEGa0mfRPJXVEcZrPkeHSKDFcuRQMuB/wZmmNkX3X27u99HeOGNyKawu//F3be4+0eED9vu9XujLBxueECW9be7+yfRXQM+Bv7m7tVmdipwtcV4zqa7f9/dr45u/wLoRXwXyfsU2D86bO4cYAGwZxxb4d39JcLejh+4++0eDoG+E9iJ8ObJtv5awjnco4FB0WNvEVaKu2ZbP6VPXfT7N4RzxyfHtKcOd19COBd9OeEDEnf/PWEZx3Fu+v3Abwihpoe7T3f3W4F+FtOhve6+nWjco/ufuPtYYLfUreRZtKgDLnP3SYRrG3wPqDCzzqkDZbEVvjtwc/R6hTAtY8xsj5i27HcG5rj7C2bWlxDCZwE/NrOfQvaHq1o4xQPCCmkpIdhebmbXAgtj2APfnbDyq4v25jwIXAlcH9c0EP5R2dHM9iAcmTCOsKflgah+NqdJ9ADOcfdV7l5NCIHTLKZTR1N8AGBm5YTPpuMJh1YvhaznkQGbCUcp4O5/JhxBMNrMfpzNSEuHk1j+guLKYIWav6A4MliS+SuqW9AZLAf5C5TBmpSD/AXKYC1V/Bksl1uckvwhrIjOI7ywzwJOB6qA3WLuswvwC+BPwBvAwASm5S7gB4Q9CnEeKWJp96dGPfrH2ONK4M/AV6L7E4A9Elrm9eMfyzImfLifRtjb9Y3oZyWwb4LjvwIoibnuMdFr9EjCHor/A/aOsX6nlNunAc8DPbOsOSTl9nRgLbBnymP1e4EPjKF+n5Tb84FfAQdH99v0fkurX78HuYSwsvoVn+1pHhxD/RLCXtjz+GwP0UDgGeDwOJZBdH8QcH90+5uEa1m0ec9y2jSMARYC6wj/sBjhn6GniY4oyLL+CMLRCZcBF6c8/gJwQRvr7592v36v8qio116pj8cwDXOBh4EXgR+mPP4i4boQ2db/T0JQO5Fw2PZNhD3vtwM7tnUa9NPxfshR/op6FWQGo8jyV9o0FFwGI6H8FdUuqAxGwvkrQw9lsCbqR/djzV8ZpkEZrPlpKPoMlniDXP4QzkOfQDiE7i7gCwn1uYhkDiW1aBrWEVbkbfqwakGfboSVaxUwLObaewAVKfdjOXwxw3w6k7AXoSyB+iOBa4Afx72MM/R6iBjDQVRzR+AC4H8IhzNmfRhyI33ql0FW84iwd3Er8EDKY1cB7/L5IPIAMDqL+venPNY15fZ84D7CntM1pBzWm8X416+cOgFPEg6rnhGtqHbKdvyjx7ul3b8DOCzLZXBfymM7Ec6HPilazpcTrg3x9ZiW8Sjga2nD3QUcEtMyPofwz8RPiUIn8C2iUw1iqN85bd7/qq3vgSbmUQ9CyD8i5bEfAidkUf/BlMcujJbrtUCX6LFlwIBspkU/He+HHOWvqFfBZjAKPH+lzKeCz2AkkL+iugWTwRpZ78SWv9J6KIM1XT+R/NXEPFIGa34eFXUGs6hhUYkOl3OP6ZsR0mrvRFhxXOLua+KuH/WYSbhYV1VC9bsQzuNf5+6vJdTDPKEXV3RI53jgr+7+pyR6JC3J+ZPSoxdh7+MnzQ7ctvp7ET603mx24MZr9CQc8voIcBhhpToteu4qwl60mwl7oqYDx7r721nU7+zu06Pnurn7P6PbzwJDCNceeDmm+iWEsHw/sAkoJ5wT3eKv2G2mfmd3r41uTyF8bewJ7v5OS+u3oMcCwj9cp7j7Ugtf4/t+a5Z5hvpd3f2U6Lnu7v5pdHsq4Vt5WjUNzdSfRTic+teEUHUSIfS0+HOjJa8hM9uFcHrB9e6+oqW1WzgNpxNC+QmE19C5hCD4ehb1G95nacNNJwS34z2cfiPSKknmr6h+QWewQs9f9fUp4AyWi/wV9WnXGSzp/NVID2WwltfPOn810kMZrHXTULQZrCg3AiXNzEo97WrqMdfPyQpKpD0ws90JX6daSrgIZE1KEPka4cr/FcANHq4bkG39f9SvQKLnhxDOiZ7pn51HHmf9xwjh5mttCf1N1Y/+oTiLsEfw9LbMn0Z6bHP3U6JrJezn7q9n87mUof4/3f3UlOdPJ1x474yYlnHqa2gsIXyMBu6JexlEz/cgfKvQQnf/a2vrN9KjYR6Z2XxgKGFv5ryY5tG2lJDTmXAtiysJF4tc3ZZpEMkFZTCReCSdvxrpoQzWdP1Y81cjPZTBmu9R9BlMG4FEpN2w8PWztxE+HKeZWRmwpbVHt7Sg/qfuPt3CBd96A6/EsdU9Q/3BhG+Bubc1e59aUf8A4Cjgv7I5IquZHuWEleGrCdUfSjiN5DceLgQaV/3619BwYKO7v59t7bT69eNfSTive0NcRz+k9Khx95PNbB8+e51ui7F+/TQMI5yL/mJbA5SIiBSupPNXWg9lsJbVjzV/NdJDGazxHkWdwbQRSETaleiwzh8RDpksIVxo770E6h8a1R/v7n9JoP6Y6KFx7v5hAvUPIxzq/MW4VxoZ5tGEhJZB/TSMd/cPEqqf9Guoc9z103qMIcyjJJdBJ2J+H4iISGFJet2Z1kMZrOn6ieSvtB7KYM33KNoMVixfES8iRSLaG7QG6EM4fDfWD/aU+jsCU+L+0E2p3xuYGmf4SKvfJ6of+16DDPMoqWVQPw2xhY8M9ZN+DcVeP61Hb5JfBrG/D0REpLAkve5M66EM1nT9RPJXWg9lsOZ7FG0G00YgEWlXLFz481jgSG/FBQJVv3B6qH7+e+RiGkREpHBo3Vb89XPRo9Dr56JHe8hgOh1MRNqdHFz4U/Xz3EP1898jF9MgIiKFQ+u24q+fix6FXj8XPfKdwbQRSERERERERESkA9DpYCIiIiIiIiIiHYA2AomIiIiIiIiIdADaCCQiIiIiIiIi0gFoI5CIiIiIiIiISAegjUAiIiIiIiIiIh2ANgKJiIiIiIiIiHQA/w+OECFG+C2q1AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x648 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "figure = plt.figure(figsize=(20,9))\n",
    " \n",
    "subplot = figure.add_subplot(1, 2, 1)\n",
    " \n",
    "plt.plot(history_original.history['accuracy'], color='blue', label='Original data')\n",
    "plt.plot(history_datagen.history['accuracy'], color='red', label='Datagen')\n",
    "plt.ylim(0.0,1.09)\n",
    "\n",
    "number_of_epochs = 20\n",
    "tick_marks = np.arange(number_of_epochs)\n",
    "plt.xticks(tick_marks, range(number_of_epochs), rotation=45)\n",
    "plt.title(\"non_augmented accuracy\", fontsize=13, \n",
    "             fontweight=0, color='black', style='italic', y=1.02)\n",
    "plt.legend(loc='lower right')\n",
    " \n",
    "\n",
    "subplot = figure.add_subplot(1, 2, 2)\n",
    "plt.plot(history_original.history['val_accuracy'], color='blue', label='Original data')\n",
    "plt.plot(history_datagen.history['val_accuracy'], color='red', label='Datagen')\n",
    "plt.ylim(0.0,1.09)\n",
    " \n",
    "tick_marks = np.arange(number_of_epochs)\n",
    "plt.xticks(tick_marks, range(number_of_epochs), rotation=45)\n",
    "plt.title(\"augmented accuracy\", fontsize=13, \n",
    "             fontweight=0, color='black', style='italic', y=1.02)\n",
    "plt.legend(loc='lower right')\n",
    " \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Compare Scores on the Holdout Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10240, 28, 28, 1)\n",
      "(10240,)\n",
      "(10240, 10)\n"
     ]
    }
   ],
   "source": [
    "# Import Holdout Dataset\n",
    "holdout = pd.read_csv('Dig-MNIST.csv')\n",
    "\n",
    "# Convert features and target into arrays\n",
    "holdout_label = holdout['label'].values\n",
    "holdout_target = to_categorical(holdout_label)\n",
    "holdout_features = holdout.loc[:,holdout.columns != 'label'].values\n",
    "holdout_features = np.true_divide(holdout_features, 255)\n",
    "holdout_features = holdout_features.reshape(-1,28,28,1)\n",
    "print(holdout_features.shape)\n",
    "print(holdout_label.shape)\n",
    "print(holdout_target.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With Data Augmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2 ... 7 2 9]\n",
      "(10240, 10)\n",
      "(10240,)\n",
      "(10240, 10)\n",
      "0.79306640625\n"
     ]
    }
   ],
   "source": [
    "# Predict accuracy on the whole holdout dataset using the different models \n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "holdout_y_pred = model2.predict_classes(holdout_features)\n",
    "\n",
    "print(holdout_y_pred)\n",
    "print(holdout_target.shape)\n",
    "print(holdout_y_pred.shape)\n",
    "\n",
    "holdout_y_pred_cats = to_categorical(holdout_y_pred)\n",
    "print(holdout_y_pred_cats.shape)\n",
    "\n",
    "print(\"Accuracy Score:\", accuracy_score(holdout_target, holdout_y_pred_cats))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Without Data Augmentation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2 ... 7 8 9]\n",
      "(10240, 10)\n",
      "(10240,)\n",
      "(10240, 10)\n",
      "0.75771484375\n"
     ]
    }
   ],
   "source": [
    "y_pred = model_original.predict_classes(holdout_features)\n",
    "\n",
    "print(y_pred)\n",
    "print(holdout_target.shape)\n",
    "print(y_pred.shape)\n",
    "\n",
    "y_pred_cats = to_categorical(y_pred)\n",
    "print(y_pred_cats.shape)\n",
    "\n",
    "print(\"Accuracy Score:\", accuracy_score(holdout_target, y_pred_cats))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "\n",
    "By using a CNN, I was able to bring up the accuracy at least 15%. Data augmentation improved the CNN accuracy by about 4%."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
